Multiprocessor type time varying image encoding system and image processor

Abstract: A multiprocessor type time varying image encoding system having a plurality of digital signal processor (DSP) modules (DMM's) connected in parallel, each DMM having a DSP, a local memory and an interrupt control unit, a plural­ ity of common memories for storing data which is being processed, parameters, etc., an input frame memory which enables reading and writing operations to be executed asyn­ chronously, a combination of a task control unit and a task table for distributing tasks to the DMM's, a plurality of independent common buses, and a combination of a bus control unit and a bus control table for bus sharing control.

Description: The present invention relates to an image processor. Fig. 3 is a block diagram of a conventional image processor disclosed, for example, in Japanese Patent Public Disclosure No. 62-86464. This image processor is arranged such that one picture frame is divided into a plurality of sectional frames #1 to #3, as exemplarily shown in Fig. 4, and unit processors (i.e., unit signal processors) are assigned to the sectional frames #1 to #3, respectively, thereby parallel-processing image signals by a plurality of unit processors, and thus achieving high efficiency encoding of time varying image signal (e.g., television signals). In Fig. 3, reference numeral 191 denotes an input bus for a image signal, e.g., a television signal, 192 a feedback bus for coded/decoded partial frame signals, 193 an output bus for outputting the result of encoding, and 194a to 194c unit processors which are assigned the sectional frames #1 to #3, respectively. Each of the unit processors 194a to 194c has a fetching unit 195, a processing unit 196 and an output unit 197. The fetching unit 195 fetches and stores an input image signal (partial frame signal) for the assigned sectional frame region from the input bus 191 and a coded/ decoded signal (described later) for neighbor processing from the feedback bus 192, in synchronism with a fetching instruction for the assigned sectional frame region. It should be noted that, as an example of the neighbor processing, a technique disclosed in Japanese Patent Public Disclosure No. 62-266678 is known. The processing unit 196 subjects the stored image data to a processing such as an encoding/decoding processing. The output unit 197 sends to the output bus 193 a coded signal as the result of the processing executed in the processing unit 196 and also sends the above-described coded/decoded signal to the other unit processors through the feedback bus 192, as being an input image auxiliary signal, in synchronism with a subsequent fetching signal. Figs. 5 and 7 show the relationship between the signal fetching time and the signal processing time in each of the unit processors 194a to 194c with regard to the signals on the buses. In the figures, the unit processors which are in charge of the sectional frames #1 to #3 are represented by #1 to #3, respectively, for simplification of the explanation. Input partial frame signals S1 to S3 as being television signals corresponding to the sectional frames A to C flow on the input bus 191 temporally sequentially, as shown in Fig. 5. The unit processor 194a, for example, fetches the input partial frame signal S1 for #1 from the input bus 191 and stores it into the fetching unit 195 in synchronism with a fetching operation timing such as that shown in Fig. 5. The input partial frame signals S1 to S3 are inputted at a constant speed, i.e., F (natural number) frames per second. Accordingly, the processing of the fetched input partial frame signals S1 to S3 must be completed before subsequent input partial frame signals S1 to S3 are fetched. The coded signals that are obtained as the result of the processing are outputted to the output bus 193 at the same time as the subsequent fetching operation. In the motion compensating interframe encoding method that is often adopted as a high efficiency image encoding technique, encoding is effected by determining the difference between an input image P and an image Q in the decoded frame which immediately precedes the present frame, the image P being separated from the image Q on the frame by an amount corresponding to the motion, as shown in Fig. 6. Accordingly, the encoding processing necessitates a coded/decoded frame the area of which is enlarged by an amount corresponding to the motion. For such an encoding processing, data for a wider area than the area covered by each of the input partial frame signals S1 to S3 is needed. Further, the coded/decoded partial frame signals F1 to F3 are outputted to the feedback bus 192 from the output units 197 and fetched into the fetching units 195 in synchronism with the fetching timing shown in Fig. 5. At this time, the fetching time lengthens by the time t required to fetch data for the excess of the area covered by each of the input partial frame signals S1 to S3. Thus, the encoding processing is executed by fetching the coded/decoded partial frame signals F1 to F3 for wider areas than the assigned sectional frames, and the resulting signals 01 to 03 are outputted to the output bus 193. In the conventional image processor having the above-described arrangement, a kind of pipeline processing is executed on the assumption that the processing time of the unit processors 194a to 194c falls within a predetermined time 1/F. Accordingly, in an image processing such as high efficiency encoding, the processing time varies depending upon the kind of input image, and the number by which the picture frame is divided must be set on the basis of the longest processing time, as described above. Even if the average processing time is much shorter than the longest processing time, the number by which the picture frame is divided cannot be reduced and, after all, a large number of unit processors must be prepared. In the arrangement shown in Fig. 5, the sum total of the time required for the unit processors 194a to 194c to fetch partial image signals and the time required to execute processing, for example, encoding/decoding processing, for the partial image signals is not greater than the period of inputting one picture frame from the input bus 191. Accordingly, the above-described processing can be continued without delay. However, in such a case that a part of one picture frame, for example, the region #2 has more motion than the other frame regions, the processing time in the unit processor 194b, which is in charge of the region #2, becomes longer than the processing time in the other unit processors 194a and 194c, as shown by the hatching in Fig. 7, and the unit processors 194a and 194c undesirably have a waiting time. Thus, since in the prior art the sectional frames assigned to the unit processors 194a to 194c are contiguous with each other, if the processing in one unit processor takes a relatively long time because a picture frame has a partiality in the nature (i.e., a large difference in the amount of data to be processed), the other unit processors are adversely affected, resulting in a lowering in the processing capacity, even if the image processor has an overall capacity sufficient to process the input image signal within a predetermined period of time. This problem may be solved by narrowing the sectional frame assigned to each unit processor and increasing the number by which one picture frame is divided. Such a solution, however, gives rise to another problem, that is, a rise in the cost of the image processor because of an increase in the number of unit processors used. It is an object of the present invention to provide an inexpensive image processor which is designed so that the number by which a picture frame is divided is determined on the basis of an average processing time, and if the processing time in a particular processing is longer than the average, the input speed of input partial frame signals is forcibly lowered, thereby enabling a reduction in the number of unit processors used. To attain the object, the present invention provides an image processor having a plurality of unit processors each including a fetching unit for fetching input partial frame signals inputted to an input bus on a frame-by-frame basis, a processing unit for executing an encoding/decoding processing on the input partial frame signal fetched in the fetching unit, and an output unit for outputting the result of the encoding/decoding processing in the processing unit, wherein the improvement comprises a storage unit for storing coded/decoded partial frame signals obtained as the result of the encoding/decoding processing, a control unit for controlling the fetching, processing, storage and output operations in each of the unit processors, and a common storage unit for storing, when the coded/ decoded partial frame signals are stored into the storage unit, a coded/decoded partial frame signal which is required to process a subsequent frame, the common storage unit being accessible from the relevant unit processor and at least one of the other unit processors. In the image processor according to the present invention, one picture frame is divided into a plurality of sectional frames, which are processed by unit processors for their exclusive use. Each unit processor stores the coded/decoded partial frame signals in the storage unit provided therein and, at the same time, stores a coded/ decoded partial frame signal which needs to be referred to from other unit processors into the common storage unit which can also be accessed by the other unit processors, thereby enabling another unit processor to use the coded/ decoded partial frame signal written in the common storage unit during an encoding processing. Thus, the number by which one picture frame is divided into sectional frames is determined on the basis of an average processing time, and when the processing time of a particular processing is longer than the average, the input speed of input partial frame signals is lowered, thereby enabling a reduction in the number of unit processors used. An image processor according to the invention is set out in claim 1. Fig. 1 is a block diagram of one embodiment of the image processor according to the present invention; Fig. 2 is a timing chart showing signals at various portions in the block diagram of Fig. 1; Fig. 3 is a block diagram showing a conventional image processor; Fig. 4 shows one example of the division of a picture frame into a plurality of regions in the prior art and in the present invention; Fig. 5 is a timing chart showing signals at various portions in the block diagram shown in Fig. 3; Fig. 6 shows an operation in the motion compensating interframe encoding method; and Fig. 7 is a timing chart showing the operation of the prior art. Referring next to Fig. 1, reference numeral 86 denotes an input bus for input partial frame signals, 87 an output bus, and 80A, 80B and 80C unit processors. Each of the unit processors 80A, 80B and 80C is provided with a processing unit 81, a fetching unit 82 for fetching input partial frame signals, a common storage unit 83 for storing some of coded/decoded partial frame signals, a storage unit 84 for storing coded/decoded partial frame signals, and an output unit for outputting a coded partial frame signal as the result of the encoding processing. A control unit 88 controls a plurality of unit processors 80A to 80C. In this embodiment, as shown in Fig. 4, one picture frame is divided into sectional frames A, B and C, to which are assigned the unit processors 80A, 80B and 80C for their exclusive use to effect processing, in the same way as in the prior art. The input bus 86 is supplied with input partial frame signals S1 to S3 in a 2 time-division manner, as shown in the timing chart of Fig. 2. The fetching unit 82 that fetches each of the input partial frame signals S1 to S3 has a double-buffer arrangement which enables writing and reading operations to be executed simultaneously. Since the input partial frame signals S1 to S3 are supplied to the input bus 86 without stagnation, one of the two buffers in each double-buffer unit is always connected to the write side so that all the input frames are always fetched. Assuming that input partial frame signals S1 to S3 for the frame m are inputted, the control unit 88 monitors the operations of the unit processors 80A to 80C, and when all the unit processors 80A to 80C complete the operation of inputting the frame m , the control unit 88 instructs all the unit processors 80A to 80C to start processing for the frame m . The time required for the unit processors 80A to 80C to complete processing depends on the input partial frame signals S1 to S3. In the illustrated example, the processing executed in the unit processor 80A for the frame m takes the longest time. Upon completion of the processings in all the unit processors 80A to 80C, the control unit 88 sequentially instructs the unit processors 80A to 80C to output the coded signals from the respective output units 85 to the output bus 87. At the same time, the control unit 88 monitors the state of the frame m+1 being inputted. Upon completion of the inputting of the frame m+1 into all the unit processors 80A to 80C, the control unit 88 instructs all the unit processors 80a to 80C to start processing for the frame m+1 , thereby executing a processing operation similar to the above. In this example, the processings executed in the unit processors 80A and 80C for the frame m+1 are longer than the period of inputting the input partial frame signals S1 to S3. As a result, when all the unit processors 80A to 80C complete the processing operations, inputting of the frame m+2 into all the unit processors 80A to 80C has already been completed, and the processing for the frame m+2 is therefore started immediately. The processing that is executed inside each of the unit processors 80A to 80C will next be explained with the unit processor 80A being taken as an example. Explanation will relate to the time at which the processing for the frame m begins. It is assumed that, at this time, the coded/decoded partial frame signal corresponding to the sectional frame A of the frame m-1 has already been stored in the storage unit 84 and the coded/decoded partial frame signal corresponding to the sub-region 2a (shown in Fig. 4), that is, the hatched portion of the sectional frame B adjacent to the sectional frame A, has also already been stored in the common storage unit 83. The control unit 88 fetches the input partial frame signals S1 to S3 from the fetching unit 82 in the form of a block to effect motion compensating interframe encoding using the coded/decoded partial frame signals for the immediately preceding frame stored in the storage unit 84 and the common storage unit 83 and outputs the resulting coded output to the output unit 85. At the same time, the control unit 88 stores the resulting coded/decoded partial frame signal into the storage unit 84. At this time, since the partial signal corresponding to the sub-region la shown in Fig. 4 is needed for the unit processor 80B to process the subsequent frame, it is stored into the common storage unit 83, which can also be accessed by the unit processor 80B, simultaneously with the storage of the coded/decoded partial frame signal. By virtue of the above-described operation, one unit processor can use the result of the processing executed in another unit processor for the processing of the subsequent frame. Moreover, in the present invention, even when the processing time for a particular frame is longer than the period of inputting the input partial frame signals S1 to S3 as in the case of the processing for the frame m+1 shown in Fig. 2, the excess processing time is absorbed by the processing time for the frames m and m+2 adjacent thereto, so that the processing time periods for these frames are leveled out. Accordingly, even in a use application in which the processing time may become longer than an average, the processing can be properly effected without the need to increase the number of unit processors used, by forcibly reducing the speed of inputting the input partial frame signals S1 to S3. Thus, it is possible to execute image processing at a reduced cost. Although in the foregoing embodiment one common storage unit 83 is disposed between each pair of adjacent unit processors, i.e., between 80A and 80B, and between 80B and 80C, a single common storage unit 83 may be provided for three or more unit processors 80. Thus, according to the present invention, when one unit processor stores a coded/decoded partial frame signal into a storage unit, a signal corresponding to a region of a picture frame which is needed for the processing executed by another unit processor is stored into a common storage unit which can also be accessed by the second unit processor. Accordingly, even in a case where the processing time may become longer than the period of inputting partial frame signals, the processing can be properly effected without the need to increase the number by which the picture frame is divided. Thus, it is possible to reduce the number by which the picture frame is divided in comparison to the prior art and hence possible to reduce the overall cost of the image processor.

Claims: An image processor for processing images consisting of frames, each frame being divided in sectional frames, comprising a plurality of unit processors (80A---80C) each including a fetching unit (82) for fetching input partial frame signals inputted to an input bus on a frame-by-frame basis, a processing unit (81) for executing an encoding/decoding processing on the input partial frame signal fetched in the fetching unit (82), and an output unit (85) for outputting the result of the coding/decoding processing in the processing unit (81), characterized by a storage unit (84) for storing coded/decoded partial frame signals obtained as the result of said processing, a control unit (88) for controlling said fetching, processing, storage and output operations in each of said unit processors (80A---80C), and a common storage unit (83) for storing, when said coded/decoded partial frame signals are stored into said storage unit (84), a coded/decoded partial frame signal corresponding to a subregion (la, 2a, 2b, 3a) which is required to process a subsequent frame, said common storage unit (83) being accessible from the relevant unit processor (80A---80C) and at least one of the other unit processors (80A---80C), whereby the number of sectional frames into which a picture frame is divided is determined on the basis of an average processing time, and if a particular processing time is longer, the input speed of input partial frame signals is lowered.

Inventors: ASANO KENICHI, SUZUKI RYUTA, ASANO, KENICHI, SUZUKI, RYUTA, ASANO, KENICHI C/O TSUSHIN SYSTEM KENKYUSHO, SUZUKI, RYUTA, C/O TSUSHIN SYSTEM KENKYUSHO
Applicants: MITSUBISHI ELECTRIC CORP, MITSUBISHI DENKI KABUSHIKI KAISHA
Application year: 1990
Main classification: H04N7
IPC codes: G06T1, G06T9, H04N7