Parallelization compile method and system

In order to make parallel processing of a serial execution type user program automatically and at a high speed without re-coding, an object code is parallelized by detection of the possibility of parallel execution in an iteration unit of a loop, detection of the possibility of parallel execution of each statement in the loop, the interchange of an outer loop by an inner loop of a multiple loop, reduction of the multiple loop to a single loop, inclined coversion for making parallel execution along a wave front plane (line) when sufficient multiplicity is not derived, and the program which is estimated to have the shortest processing time is selected from the granularity, and multiplicity of the object code, the variance of the number of instructions and the proportion of synchronization control at the time of parallelization of the object code.

BACKGROUND OF THE INVENTION 
1. Field of the Invention 
This invention relates generally to a parallel computer system. More 
particularly, the present invention relates to a method and system for 
generating an object program suitable for executing in parallel from a 
source program described by a higher language of a serial execution type, 
or the like. 
2. Description of the Prior Art 
In a parallel processor system such as a multi-processor, it has been 
necessary conventionally for a user to describe explicitly means for 
parallelization, instructions such as actuation and synchronization of a 
task, etc., into a serial type source program as a user interface. The 
article "A Data Flow Approach To multitasking on CRAY X-MP Computers", 
ACM-0-89791-174-1-12185-0107, describes the operation of multitasking by 
operating four vector processors in parallel and a user's directive method 
for that purpose. According to this prior art reference, a library for the 
control of actuation and synchronization of tasks is prepared in the 
system and a user makes a statement for calling it in a FORTRAN program. 
At a finer level the user must instruct the parallelization means for each 
loop in the form of a control statement of a comment system to a compiler. 
However, there have been no prior art references that mention automatic 
parallelization means from a serial type compiler language for such a 
multi-processor system. 
The prior art technique described above does not pay any consideration to 
automatic parallelization for hardwares capable of parallel processing. 
Therefore, it has been necessary for the user to consider by himself means 
for parallel processing and to explicitly put it into the program as 
program conversion. In other words, serial type programs that the user 
keeps as the property cannot be subjected as such to parallel execution 
and re-coding for the parallel processing and its debugging must be 
carried out. Instructions for parallelization must be changed whenever the 
characteristics of the hardware change in order to fully utilize the 
resources and the program does not run in other systems. Thus the 
versatility of the user program is lost. 
SUMMARY OF THE INVENTION 
It is therefore an object of the present invention to provide a 
parallelization compile method and system which can mitigate the load to 
users, can parallelize automatically existing serial type programs as such 
without modification and can generate efficient object codes without 
taking fine characteristics of hardwares into specific consideration even 
when coding is made afresh. 
In a compiler for generating an object code consisting of instruction 
columns for executing in parallel from a source program on a 
multi-processor consisting of processors that operate in parallel with one 
another, the present invention allots an executable parallel processing 
detected by at least one of the following two detections to any of a 
plurality of processors described above, whereby one of the detection is 
to detect the possibility of parallel execution in each iteration unit 
inside a certain loop for each of the loops for making iteration 
computation in a program for a computer of a serial execution type single 
processor system and the second of the detection is to detect the 
possibility of parallel execution of each statement inside a loop; then 
estimates an elapsed time before the computation result is obtained (or 
execution time) when the object code described above is executed in 
parallel, in accordance with the performance of the allotted processors; 
and selects the processing which is judged to be the best at the time of 
compile inclusive of the estimated result and the estimated time when 
parallel execution is not made; so that a serial execution type program is 
converted to a parallel execution type program. 
The above will be explained in further detail. 
In automatic parallelization processing of a complier, the object of the 
invention, can be accomplished by executing the following processings from 
outer loops to inner loops. 
First of all, the possibility of parallel execution of each iteration of 
the loop is detected. 
##EQU1## 
For the program such as described above, the possibility of parallel 
execution of inner loop DO 20 at each value of I=1, I=2 , . . . , I=N is 
detected, and 
##EQU2## 
Next, the possibility of parallel execution of the statement (1), (2) at 
each value of I=1, I=2 , . . . , J=M is detected. 
##EQU3## 
In the manner described above, the program is divided and the items that 
will affect improvement in a speed, that is, an acceleration ratio, when 
the programs are executed and processed sequentially in parallel by 
mutually different processors are calculated and the results are stored in 
a table for each loop. First of all, multiplicity is calculated by judging 
whether or not the definition determining the data value on the memory at 
the same address and the use of such a value occurs in the different 
number of iterations. Next, the number of instructions per parallel 
processing unit (which is referred to as "granularity") and its variation 
are calculated by counting the kinds and number of textual operators of 
the source program and the estimated number of iteration of loops. 
Furthermore, the proportion (ratio) of the number of instructions 
necessary for executing in parallel for each iteration of loop to the 
total number of instructions is calculated. 
Next, the possibility of parallel execution for each statement is detected 
In the case of the same source program as one described above, for 
example, the statement (1) and the statement (2) are divided by the inner 
loop and the possibility of their parallel execution is detected. 
##EQU4## 
Since two statements exist inside the loop in this example, the program is 
divided into two and the items that will affect the speed-up, that is, the 
acceleration ratio, when they are executed in parallel by different 
processors are calculated and stored in the table for each loop. First, 
loop-unsplittable recursive calculation is detected and the multiplicity 
is calculated from the number of statements Next, the number of 
instructions (granularity) is counted for each statement as the unit of 
parallel processing, for each inner loop and for each recursive 
calculation unit from the number of textural operators of the source 
program and the estimated loop iteration times to compute the size of each 
granule and the variance (or inequality) of granularity. Furthermore, the 
proportion of the number of instructions for making synchronization 
control to the total number of instructions is computed from the data 
dependence relation of each statement. 
Furthermore, the new possibility of parallel execution that occurs due not 
to one loop alone but to the combination of multiple loops is detected. 
First, in order to judge whether or not the multiple loops can be 
combined, judgement is made as to whether or not they can be converted to 
tightly nested multiple loops. Only when such a possibility is recognized, 
the possibility of parallel processing when the following three kinds of 
loop structure conversion are made is examined. 
When there is the possibility of parallel processing for the loop as the 
object, loop conversion is made in order to interchange it by its outer 
loop to improve granularity. Therefore, only when this condition is 
satisfied, loop interchange possibility is judged and if it is found 
possible, the multiplicity, the variance (or inequality) of the number of 
instructions and the proportion of synchronization control are calculated 
and the results are stored in the table. In the case of the source program 
which is the same as described above, interchange is made between the DO 
10 loop and the DO 20 loop and the possibility of parallel execution of 
the inner DO 10 loop is detected at each value of J=1, J=2 , . . . , J=M 
of the DO 20 loop which now becomes a new outer loop. 
Then, the program is divided in the following way. 
##EQU5## 
The multiplicity, the variance of the number of instructions and the 
proportion of synchronization control when they are executed by different 
processors are computed. 
When parallel processing can be made for the object loop but multiplicity 
is not sufficient because its loop length is small, loop collapsion 
(reduction to a single loop) is made in order to reduce the outer loops of 
that loop to the single loop and to improve the multiplicity. Therefore, 
only when this condition is satisfied, the possibility of group collapsion 
is judged and if it proves YES, the multiplicity, variance of the number 
of instructions and proportion of synchronization control when reduction 
to the single loop is made are computed and stored in the table. In the 
case of the same source program as described above, the DO 10 loop and the 
DO 20 loop are reduced to the single loop as illustrated below: 
##EQU6## 
The possibility of parallel execution of the DO 10 loop at each value of 
K=1, K=2 , . . . , K=M*N is then detected. 
##EQU7## 
When sufficient multiplicity cannot be derived due to the limit of the 
dependence relation of the data or the like at one loop level alone, 
inclined conversion is made by carrying out in parallel processing along a 
wave front line (plane) to derive sufficient multiplicity. Therefore, when 
this condition is satisfied, whether or not inclined conversion is 
possible is judged and the multiplicity, variance of the number of 
instructions and the proportion of synchronization control at the time of 
inclined conversion are calculated and stored in the table. In the case of 
the source program described above, since no data dependence relation 
impeding the parallel processing exists, this conversion is not made. 
Finally, parallelization means which is judged to be the one that shortens 
most the elapsed time till acquisition of the result (execution time) when 
parallel processing is made from the multiplicity, variance of the number 
of instructions and the proportion of the synchronization control thus 
calculated is selected. The subsequent processing of compiler converts the 
serial type program to the parallel execution type program. 
The foregoing and other objects, advantages, manner of operation and novel 
features of the present invention will be understood from the following 
detailed description when read in conjunction with the accompanying, 
drawings.

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
Hereinafter, one embodiment of the present invention in a FORTRAN compiler 
for a multi-processor system wherein a plurality of processors operate in 
parallel with one another will be explained with reference to the 
accompanying drawings. 
FIG. 2 shows an example of the multi-processor to which the invention is 
applied. Here, the embodiment of the invention will be explained about the 
multi-processor system sharing a main storage by way of example, but the 
present invention can also be applied to a multi-processor system of the 
type wherein each processor has its own memory or storage. There are 
disposed CPUs 82 to 84 that operate in parallel with one another and share 
the main storage 80. Therefore, when each CPU makes access to the same 
address, sequence must be guaranteed by synchronization control. The 
detailed embodiment of the present invention for generating an object code 
which reduces the elapsed time before acquisition of the calculation 
result by utilizing effectively hardware resources will now be explained. 
FIG. 3 shows the overall structure of the compiler to which the present 
invention is applied. The compiler works in concert with a CPU 10 as 
appreciated by one of ordinary skill in the art. Parsing 13 in FIG. 3 
receives the source program 11 of FORTRAN as its input and converts it to 
an intermediate language 6. Receiving this intermediate language 6 as the 
input, middle process 14 makes optimization and parallelization to modify 
the intermediate language 6. Code generation 15 generates an object code 
12 to be executed in parallel from the intermediate language 6 modified by 
the middle process 14. The present invention relates to the middle process 
6 and reduces the elapsed time to minimum when the object code 12 is 
executed in parallel. 
In the middle process 14 shown in FIG. 3, the structure of the processing 
relating to automatic parallelization is shown in FIG. 1. The FORTRAN 
program shown in FIG. 4 is hereby used as the example of the source 
program 11 inputted in FIG. 3. The middle process 14 prepares two loop 
tables DO 10, DO 20 (the loop table 7 shown in FIG. 1) from the FORTRAN 
program shown in FIG. 4 and represents that they constitute a multiple 
loop. FIG. 5 shows the table structure for each loop. 
Means for parallel processing is determined by filling each field from the 
outermost loop to inner loops in accordance with the loop tables shown in 
FIG. 5. FIG. 6 is a PAD table showing the main control of the automatic 
parallelization processing 1. After the loop table is secured, the middle 
process 14 designates the table of the leading loop on the outermost side 
and calls the automatic parallelization processing 1. Processing is made 
for this loop as shown in FIG. 6. 
First of all, a pointer is given to the loop table 22 in FIG. 5 
corresponding to the leading loop DO 10, loop 16-21 on the outmost side in 
FIG. 4 and the processing shown in FIG. 6 is executed. The parallelism 
detection processing 2 for each iteration of the loop of FIG. 1 is called 
for the loop FIG. 5, 22 in order to detect parallelism for each iteration 
(61 in FIG. 6). 
If the processing is made sequentially from the outer loop as described 
above, new analysis is not necessary for the judgement of structure 
convertibility and for the detection of parallelism after structure 
conversion and the data that have already been used may be employed. For, 
only minimum necessary parsing is made in order to reduce the compile 
time. 
The outline of the parallelism detection processing 2 for each iteration of 
the loop shown in FIG. 1 is represented by a PAD table in FIG. 7. As to 
the DO 10 I loop 16-21 shown in FIG. 4, flow dependence exists from C(I) 
on the left side of 17 to C(I) on the right side of 20 and output 
dependence exists from C(I) on the left side of 17 to C(I) on the left 
side of 20. Furthermore, flow dependence exists from A(I+1, J) on the left 
side of 19 to A(I, J) on the right side of 20. Since dependence relating 
to C among them is loop independent, flow dependence on A is detected by 
the processing 70 in FIG. 7. The processing 71 calculates from these loop 
carried dependence iterated twice that multiplicity is 1 and this value is 
stored in the field 24 of the loop table of FIG. 5. The multiplicity "1" 
means that execution is equivalent to the serial execution. The processing 
72 in FIG. 7 estimates the dynamic number of executed instruction of DO 
10I loop per each iteration. The statement 17 becomes 1, the statement 19 
is 100 by multiplying the textural number of instruction 1 by the loop 
length 100 and the statement 20 is 100 by multiplying 1 by the loop length 
100. The sum of them 201 is stored in the fields 25 and 26 and the product 
of this value 201 by the loop length 10 of the outer loop 10 is stored in 
the field 23. Since synchronization is necessary whenever I is incremented 
by 1 from the data dependence relation, the processing 74 stores the 
dynamic number of executed instructions per each iteration of the loop I 
in the field 27. 
Next, the processing 62 shown in FIG. 6 judges that sufficient multiplicity 
does not exist because the multiplicity is 1 and the processing 65 calls 
the parallelism detection portion 3 for each statement in the loop shown 
in FIG. 1. The outline of this processing is shown by a PAD diagram in 
FIG. 8. The processing 75 shown in FIG. 8 detects the loop split point 
between the statement 17 and the statement 18 in FIG. 4 and stores the 
multiplicity 2 in the field 28 in FIG. 5 because it is only the loop 
splitting point. As the number of executed instructions when the loop is 
split, the processing 76 stores the product 10 of the value 1 of the 
statement 17 by the outer loop length 10 in the field 29 of the minimum 
value. As the executed number of instructions of the statements 18 to 20, 
the value 200 as the products of the 2 of each statement 19, 20, the outer 
loop length 10 and the inner loop length 20 is stored in the field 30 of 
the maximum value. The processing 77 judges from the flow dependence for 
the statements 17 to 20 that synchronization exists ten times for the 
instruction 2000, and stores it in the field 31. Thereafter the processing 
returns to 63 in FIG. 6. 
Since there is no outer loop for the DO 10I loop in FIG. 4, the processing 
65 is not executed but the flow proceeds to the processing 66, where 
parallelization means selection processing 5 in FIG. 1 is called. The 
outline of this processing is shown in a PAD diagram in FIG. 9. In the 
case of the DO 10I loop, the speed of serial execution is higher than that 
of any other systems, the processing 92 selects it and fills the fields 33 
and 34. Therefore, the fields become such as 23'-34' shown in FIG. 11. The 
processing 69 shows this time the table 38 of the DO 10J loop through the 
judgement of the processing 67 in FIG. 6 and again executes the processing 
shown in FIG. 6. 
Since no carried dependence relating to the loop exists for the DO 10J loop 
from the statement 18 to the statement 21 in FIG. 4, the processing 71 in 
FIG. 7 stores the loop length 100 in the field 40 in FIG. 5. Furthermore, 
the processing 73 and 74 store the number of instructions 2 per loop in 
the fields 41, 42 and the product 2000 obtained by multiplying the loop 
length 100 of the inner loop and the loop length 10 of the outer loop is 
stored in the field 39. Since synchronization control is necessary for 
each outer loop due to the dependence on the outer loop, the information 
that the number of synchronization controls is 1 per 200 instructions is 
stored in the field 43. The processing 62 shown in FIG. 6 judges that 
there is already sufficient parallelism due to parallelization of the 
number of times of iterations. Therefore, the processing 63 is not 
executed but the processing 65 calls the loop structure convertibility 
judgement processing 4 in FIG. 1 through the processing 64. The outline of 
this processing is shown in FIG. 10. 
Since tightly nested multiplication is possible with this outer loop in the 
DO 10J loop of the statements 18 to 21 in FIG. 4, the flow proceeds from 
the processing 100 to the processing 101. Here, it is known that 
sufficient parallelism can be obtained by parallel execution for each 
iteration of the loop, the processing 102 examines the possibility of loop 
interchange in order to improve granularity. If the result proves YES, the 
processing 104 prepares the tables 120 to 127 of FIG. 11 so as to 
represent the multiplicity, the number of instructions and synchronization 
control when the loop interchange is made, and the estimated values 
described above are stored. The multiplicity does not change and the loop 
length is 100 (in field 123 of FIG. 11). However, the instruction 
granularity changes to 20 due to the loop interchange (fields 124, 125) 
and the synchronization control becomes unnecessary during the operation. 
Accordingly, the synchronization of once per 2000 instructions is stored 
in the field 126 in the sense that it is made once after all the 
instructions are complete If there is the loop carry dependence, the 
processings 109 and 111 examine the possibility of converting the loop by 
wave front line (plane) and calculate the multiplicity, the instruction 
granularity and synchronization control at that time and store them in the 
tables. If the loop length is short and the multiplicity and the 
instruction granularity are small even when the data dependence relation 
is independent, the processings 106 and 108 examine the possibility of 
loop collapsion (reduction to the single loop), calculate the multiplicity 
and synchronization control at that time and stores them in the tables. 
Therefore the flow proceeds to the processing 66 in FIG. 6, which calls 
the parallelization means selection processing 5 of FIG. 1 and the 
judgement is made as shown in FIG. 6. In the processing 92, the 
instruction that each iteration is executed in parallel is stored in the 
field 49' in FIG. 11 and the processings 94 and 95 make the loop 
interchange from the results of the tables designated by the field 48' and 
select means for executing the iteration times of the DO 10J loop. The 
processing 97 stores the pointer of the tables of the fields 120 to 127 in 
the field 49' in FIG. 11 and the processing 59 sets the number of 
processors to NPE and stores the estimated elapsed time 20.times.100/NPE 
in the field 50' because no synchronization control exists. 
When the detection, evaluation and selection of parallelism for the FORTRAN 
program shown in FIG. 4 are completed in this manner, the loop table 7 
shown in FIG. 1 changes to the table shown in FIG. 11. From this table the 
processings subsequent to the middle process 14 in FIG. 3 convert the 
program shown in FIG. 4 to the intermediate language 6 as shown in FIG. 
12. Each iteration is executed in parallel for the loop 131. Thus the code 
generation processing 15 in FIG. 3 generates the object code 12, and the 
mode during the execution of this object code is shown in FIG. 13. Each 
iteration of the loop 131 in FIG. 12 is allotted to the NPE sets of 
processors and executed in parallel. 
Thus the present invention has been described in detail with reference to 
one embodiment thereof. 
Broadly speaking, the embodiment described above can be accomplished by 
software as a system by a large-scale computer or the like but can be 
accomplished hardware-wise by using micro-processors or mini-computers and 
necessary storage devices for each parallelism detection function 2, 3, 
loop structure convertibility judgement 4, parallelization means section 
5, etc., shown in FIG. 1. 
In accordance with the present invention described above, conventional 
serial execution type user programs can be executed automatically and at a 
high speed by a parallel processing system without the need of re-coding 
them. At this time a high object code having high execution efficiency and 
a short elapsed time can be generated by using effectively the hardware 
resources.