Patent Publication Number: US-10771798-B2

Title: Multi-stream image processing apparatus and method of the same

Description:
BACKGROUND OF THE INVENTION 
     1. Field of the Invention 
     The present disclosure relates to a multi-stream image processing technology, especially to a multi-stream image processing apparatus and method of the same. 
     2. Description of Related Art 
     In some applications of the image processing technology, a plurality of image streams are generated according to the same image source. For example, after retrieving the images of the environment, an IP camera may generate a plurality of image streams, while some image streams having higher resolution can be displayed as higher resolution frames on a corresponding display module, and some image streams having lower resolution can be displayed on a display module having a lower resolution to provide an instant surveillance mechanism. 
     However, a plurality of corresponding circuits are required to perform image processing on a plurality of image streams. The cost of the circuits is thus increased. 
     SUMMARY OF THE INVENTION 
     In consideration of the problem of the prior art, an object of the present disclosure is to provide a multi-stream image processing apparatus and a multi-stream image processing method to perform process on a plurality of image streams by using a single latter stage circuit with a time-division method to avoid the high cost of disposition of a multiple of latter stage circuits. 
     The present disclosure discloses a multi-stream image processing method that includes steps outlined below. A plurality of image streams are generated by a former stage circuit according to a same image source, wherein the image streams comprises a main image stream and at least one sub image stream, and a resolution of the main image stream is higher than a resolution of the sub image stream. Within an image frame processing time period, a N-th sub image frame of the sub image stream is stored by the former stage circuit in at least one current sub image storage block of a memory module and a N-th main image frame of the main image stream is stored by the former stage circuit in a main image storage block of the memory module. Within a first sub period of the image frame processing time period, a N−1-th sub image frame of the sub image stream stored in a previous sub image storage block of the memory module is read by a latter stage circuit and the N−1-th sub image frame is processed by the latter stage circuit. Within a second sub period of the image frame processing time period behind the first sub period, the N-th main image frame stored in the main image storage block is read by the latter stage circuit and the N-th main image frame is processed by the latter stage circuit. 
     The present disclosure also discloses a multi-stream image processing apparatus that includes a memory module, a former stage circuit, a latter stage circuit and a processing circuit. The processing circuit is electrically coupled to the memory module, the former stage circuit and the latter stage circuit, and is configured to execute a plurality of software or firmware executable commands to execute a multi-stream image processing method. The multi-stream image processing method includes steps outlined below. A plurality of image streams are generated by the former stage circuit according to a same image source, wherein the image streams comprises a main image stream and at least one sub image stream, and a resolution of the main image stream is higher than a resolution of the sub image stream. Within an image frame processing time period, a N-th sub image frame of the sub image stream is stored by the former stage circuit in at least one current sub image storage block of a memory module and a N-th main image frame of the main image stream is stored by the former stage circuit in a main image storage block of the memory module. Within a first sub period of the image frame processing time period, a N−1-th sub image frame of the sub image stream stored in a previous sub image storage block of the memory module is read by the latter stage circuit and the N−1-th sub image frame is processed by the latter stage circuit. Within a second sub period of the image frame processing time period behind the first sub period, the N-th main image frame stored in the main image storage block is read by the latter stage circuit and the N-th main image frame is processed by the latter stage circuit. 
     These and other objectives of the present disclosure will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiments that are illustrated in the various figures and drawings. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a block diagram of a multi-stream image processing apparatus in an embodiment of the present invention. 
         FIG. 2  is a flow chart of a multi-stream image processing method in an embodiment of the present invention. 
         FIG. 3  is a timing diagram of the image processing performed by the multi-stream image processing apparatus in an embodiment of the present invention. 
         FIG. 4  is a block diagram of the synchronization circuit in an embodiment of the present invention. 
         FIG. 5  is a flow chart of a synchronization flow performed during the operation of the synchronization circuit in an embodiment of the present invention. 
         FIG. 6  is a diagram of the main image frame in an embodiment of the present invention. 
         FIG. 7  is a block diagram of the synchronization circuit in another embodiment of the present invention. 
         FIG. 8  is a flow chart of a synchronization flow performed during the operation of the synchronization circuit in another embodiment of the present invention. 
         FIG. 9  is a timing diagram of the image processing performed by the multi-stream image processing apparatus in another embodiment of the present invention. 
         FIG. 10  is a block diagram of the synchronization circuit in yet another embodiment of the present invention. 
         FIG. 11  is a flow chart of a synchronization flow performed during the operation of the synchronization circuit in yet another embodiment of the present invention. 
     
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
     An aspect of the present invention is to provide a multi-stream image processing apparatus and a multi-stream image processing method to perform process on a plurality of image streams by using a single latter stage circuit with a time-division method to avoid the high cost of disposition of a multiple of latter stage circuits. 
     Reference is now made to  FIG. 1 .  FIG. 1  is a block diagram of a multi-stream image processing apparatus  100  in an embodiment of the present invention. The multi-stream image processing apparatus  100  is able to retrieve the images of an environment to generate a plurality of image streams MS (main image stream), SS 1  (first sub image stream) and SS 2  (second sub image stream) according to the same image source IS and subsequently perform processing on the image streams MS, SS 1  and SS 2  to generate a processed main image stream EMS, a processed first sub image stream ESS 1  and a processed second sub image stream ESS 2 . 
     The multi-stream image processing apparatus  100  includes a memory module  110 , a former stage circuit  120 , a latter stage circuit  130 , a processing circuit  140  and a synchronization circuit  150 . 
     In an embodiment, the memory module  110  has different blocks to store different data required to process the plurality of image streams. The memory module  110  can be implemented by a memory having a higher speed such as, but not limited to a double data rate synchronous dynamic random access memory (DDR SDRAM). 
     In an embodiment, the multi-stream image processing apparatus  100  may further include a memory controller  115 . The memory controller  115  can be implemented by such as, but not limited to a memory interface unit (MIU). Other circuit modules in the multi-stream image processing apparatus  100 , e.g. the former stage circuit  120  and the latter stage circuit  130 , can perform read operation and write operation on the memory module  110  through the memory controller  115  to store data in the memory module  110  or retrieve data from the memory module  110 . 
     In an embodiment, the former stage circuit  120  is an image signal processor (ISP) configured to generate the plurality of image streams MS, SS 1  and SS 2  according to the same image source IS. The image source IS can be such as, but not limited to image sensing components in an IP camera. After the image sensing components sense the image, the former stage circuit  120  generates the image streams MS, SS 1  and SS 2  accordingly and stores the image streams MS, SS 1  and SS 2  having different resolutions in the memory module  110  through the memory controller  115 . 
     The image stream MS is a main image stream and the image streams SS 1  and SS 2  are respectively a first sub image stream and a second sub image stream. The resolution of the main image stream MS is higher than the resolution of the first sub image stream SS 1  and the resolution of the second sub image stream SS 2 . In an embodiment, the resolution of the main image stream MS is 4K, The resolution of the first sub image stream SS 1  is 1280×720. The resolution of the second sub image stream SS 2  is 720×576. It is appreciated that the number of the sub image streams and the resolution of each of the image streams described above are merely an example. The present invention is not limited thereto. In an embodiment, the number of the sub image stream can be one or more than one. 
     In an embodiment, the latter stage circuit  130  is an encoder configured to read the image streams MS, SS 1  and SS 2  from the memory module  110  through the memory controller  115  to perform encoding process and generate the processed main image stream EMS, the processed first sub image stream ESS 1  and the processed second sub image stream ESS 2  that are finished being encoded. In different embodiments, the encoding process is performed based on such as, but not limited to H264, H265 or other encoding standards. 
     The synchronization circuit  150  is electrically coupled to the former stage circuit  120 , the latter stage circuit  130  and the processing circuit  140  and is configured to, according to the control of the processing circuit  140 , balance the speed that the former stage circuit  120  stores the image stream to the memory module  110  and the speed that the latter stage circuit  130  reads the image stream from the memory module  110 . 
     The processing circuit  140  is electrically coupled to the memory module  110 , the former stage circuit  120 , the latter stage circuit  130  and the synchronization circuit  150 . The processing circuit  140  can execute software or firmware executable commands  141  to perform the function of the multi-stream image processing apparatus  100 . More specifically, the processing circuit  140  can retrieve the software or firmware executable commands  141  from a storage module (not illustrated) included in the multi-stream image processing apparatus  100 , in which the software or firmware executable commands  141  include such as, but not limited to firmware of the former stage circuit  120 , the latter stage circuit  130  and the synchronization circuit  150  and other related commands for operating and controlling the former stage circuit  120 , the latter stage circuit  130  and the synchronization circuit  150 . The processing circuit  140  further operates and controls the former stage circuit  120 , the latter stage circuit  130  and the synchronization circuit  150  accordingly to perform the storing, reading and encoding of the image streams MS, SS 1  and SS 2  and generate the processed main image stream EMS, the processed first sub image stream ESS 1  and the processed second sub image stream ESS 2 . 
     It is appreciated that in an embodiment, the storage module can be implemented by another memory independent from the memory module  110 . For example, the storage module can be implemented by such as, but not limited to a optical disk, a random access memory (RAM), a read only memory (ROM), a floppy disk, a hard drive or a CD-ROM. In another embodiment, the storage module can also be implemented together with the memory module  110  as a single memory. 
     Reference is now made to  FIG. 2  at the same time. The detail function of the multi-stream image processing apparatus  100  is further described in accompany with  FIG. 1  and  FIG. 2  in the following paragraphs. 
       FIG. 2  is a flow chart of a multi-stream image processing method  200  in an embodiment of the present invention. The multi-stream image processing method  200  can be used in the multi-stream image processing apparatus  100  illustrated in  FIG. 1 . In an embodiment, the multi-stream image processing method  200  includes the steps illustrated in  FIG. 2 . 
     Step S 210 : a plurality of image streams MS, SS 1  and SS 2  are generated by the former stage circuit  120  according to the same image source IS. As described above, image streams MS, SS 1  and SS 2  include the main image stream MS, the first sub image stream SS 1  and the second sub image stream SS 2 . The resolution of the main image stream MS is higher than the resolution of each of the first and the second sub image streams SS 1  and SS 2 . 
     Step S 220 : Within an image frame processing time period, a N-th sub image frame of the sub image stream SS 1  is stored by the former stage circuit  120  in the current first sub image storage block  114 A of the memory module  110 , a N-th sub image frame of the sub image stream SS 2  is stored by the former stage circuit  120  in the current second sub image storage block  116 A of the memory module  110 , and a N-th main image frame of the main image stream MS is stored by the former stage circuit  120  in a main image storage block  112  of the memory module  110 . N is a positive integer. 
     Reference is now made to  FIG. 3  at the same time.  FIG. 3  is a timing diagram of the image processing performed by the multi-stream image processing apparatus  100  in an embodiment of the present invention. 
     The time sequence during the processing of the multi-stream image processing apparatus  100  includes a plurality of image frame processing time periods, such as the consecutive three image frame processing time periods T N−1 , T N  and T N+1  exemplarily illustrated in  FIG. 3 . Each two adjacent image frame processing time periods include a synchronization signal time period therebetween, such as the synchronization signal time period TS N  between the image frame processing time periods T N−1  and T N , and the synchronization signal time period TS N+1  between the image frame processing time periods T N  and T N+1 . The synchronization signal time periods TS N  and TS N+1  respectively correspond to the transmission of the synchronization signals S N  and S N+1  and respectively mark the beginning of the image frame processing time periods T N  and T N+1 . 
     In the present embodiment, within the synchronization signal time periods TS N  and TS N+1 , both of the former stage circuit  120  and the latter stage circuit  130  do not perform data processing. The former stage circuit  120  and the latter stage circuit  130  only perform data processing within the image frame processing time periods T N−1 , T N  and T N+1 . 
     In  FIG. 3 , the time periods corresponding to the processing of the former stage circuit  120  is illustrated as blocks having slashes. Since the former stage circuit  120  can simultaneously generate the image streams MS, SS 1  and SS 2 , the former stage circuit  120  simultaneously stores the N-th sub image frame of the sub image stream SS 1  in the current first sub image storage block  114 A, stores the N-th sub image frame of the sub image stream SS 2  in the current second sub image storage block  116 A and stores the N-th main image frame of the main image stream MS in the main image storage block  112  after the image frame processing time period T N  begins. 
     Step S 230 : Within a first sub period T S1  of the image frame processing time period T N , a N−1-th sub image frame of the sub image stream SS 1  stored in a previous first sub image storage block  114 B of the memory module  110  is read by the latter stage circuit  130  and a N−1-th sub image frame of the sub image stream SS 2  stored in a previous second sub image storage block  116 B of the memory module  110  is read by the latter stage circuit  130 . The N−1-th sub image frame of the sub image stream SS 1  and the N−1-th sub image frame of the sub image stream SS 2  are processed by the latter stage circuit  130 . 
     In an embodiment, as illustrated in  FIG. 3 , within the image frame processing time period T N−1 , the N−1-th sub image frame of the sub image stream SS 1  and the N−1-th sub image frame of the sub image stream SS 2  are respectively stored in the previous first sub image storage block  114 B and the previous second sub image storage block  116 B by the former stage circuit  120  through the memory controller  115 . 
     The time periods corresponding to the processing of the latter stage circuit  130  is illustrated as blocks having dots. The latter stage circuit  130  is operated by using a time-division method to process one image frame in one time period. As illustrated in  FIG. 3 , within the first sub period T S1 , the latter stage circuit  130  first reads the N−1-th sub image frame corresponding to the first sub image stream SS 1  from the previous first sub image storage block  114 B and processes the N−1-th sub image frame to generate the image frame corresponding to the processed first sub image stream ESS 1 . Subsequently, within the remained time period within the first sub period T S1 , the latter stage circuit  130  reads the N−1-th sub image frame corresponding to the second sub image stream SS 2  from the previous second sub image storage block  116 B and processes the N−1-th sub image frame to generate the image frame corresponding to the processed second sub image stream ESS 2 . 
     Step S 240 : Within a second sub period T S2  of the image frame processing time period T N  behind the first sub period T S1 , the N-th main image frame stored in the main image storage block  112  is read by the latter stage circuit  130  and the N-th main image frame is processed by the latter stage circuit  130 . 
     In an embodiment, a capacity of the main image storage block  112  is larger than or equals to a size of the N-th main image frame. In some embodiments, the operation speed of the latter stage circuit  130  may be faster than the operation speed of the former stage circuit  120 . As a result, in order to avoid the occurrence of the condition that the speed that the latter stage circuit  130  reads data from the main image storage block  112  exceeds the speed that the former stage circuit  120  stores data to the main image storage block  112 , the synchronization circuit  150  performs coordination and synchronization between the former stage circuit  120  and the latter stage circuit  130 . 
     Reference is now made to  FIG. 4 .  FIG. 4  is a block diagram of the synchronization circuit  150  in an embodiment of the present invention. 
     As illustrated in  FIG. 4 , the synchronization circuit  150  includes a first comparison module  400 , a second comparison module  402  and a synchronization processing module  404 . 
     The first comparison module  400  and the second comparison module  402  can retrieve frame processing information FP 1  and FP 2  from the former stage circuit  120  and the latter stage circuit  130  respectively. The frame processing information FP 1  includes a former stage frame number F 1  and a storing row number RW of the image frame that the former stage circuit  120  currently stores. The frame processing information FP 2  includes a latter stage frame number F 2  and a reading row number RR of the image frame that the latter stage circuit  130  currently reads. 
     In an embodiment, the first comparison module  400  performs comparison according to the former stage frame number F 1  of the image frame that the former stage circuit  120  currently stores and the latter stage frame number F 2  of the image frame that the latter stage circuit  130  currently reads to determine whether the image frame that the former stage circuit  120  currently stores and the image frame that the latter stage circuit  130  currently reads are the same image frame. The second comparison module  402  performs comparison according to the storing row number RW of the image frame that the former stage circuit  120  currently stores and the reading row number RR of the image frame that the latter stage circuit  130  currently reads to determine whether the reading row number RR of the image frame that the latter stage circuit  130  currently reads exceeds the storing row number RW of the image frame that the former stage circuit  120  currently stores. 
     The synchronization processing module  404  further determines the operation condition of the former stage circuit  120  and the latter stage circuit  130  according to the comparison results of the first comparison module  400  and the second comparison module  402  to further determines the synchronization strategy being used. 
     Reference is now made to  FIG. 5  at the same time.  FIG. 5  is a flow chart of a synchronization flow  500  performed during the operation of the synchronization circuit  150  in an embodiment of the present invention. The operation mechanism of the synchronization circuit  150  is further described in detail in accompany with  FIG. 4  and  FIG. 5  in the following paragraphs. 
     Step S 510 : As illustrated in  FIG. 5 , the first comparison module  400  retrieves the former stage frame number F 1  and the latter stage frame number F 2  in the frame processing information FP 1  and the frame processing information FP 2  respectively from the former stage circuit  120  and the latter stage circuit  130  to perform comparison. 
     Step S 520 : According to the first comparison result CR 1  of the first comparison module  400 , the synchronization processing module  404  further determines whether the image frame that the former stage circuit  120  currently stores and the image frame that the latter stage circuit  130  currently reads are the same image frame. 
     Step S 530 : When the image frame that the former stage circuit  120  currently stores and the image frame that the latter stage circuit  130  currently reads are not the same image frame, e.g. the condition within the first sub period T S1  of the image frame processing time period T N , in which the latter stage circuit  130  reads the N−1-th sub image frame and the former stage circuit  120  stores the N-th main image frame, the synchronization processing module  404  does not activate the synchronization mechanism of the synchronization circuit  150 . 
     Step S 540 : When the image frame that the former stage circuit  120  currently stores and the image frame that the latter stage circuit  130  currently reads are the same image frame, e.g. the condition within the second sub period T S2  of the image frame processing time period T N , in which the latter stage circuit  130  reads the N-th sub image frame and the former stage circuit  120  also stores the N-th main image frame, the synchronization processing module  404  activates the synchronization mechanism of the synchronization circuit  150 . The second comparison module  402  retrieves the storing row number RW and the reading row number RR in the frame processing information FP 1  and FP 2  respectively from the former stage circuit  120  and the latter stage circuit  130  to perform comparison. 
     Reference is now made to  FIG. 6  at the same time.  FIG. 6  is a diagram of the main image frame  600  in an embodiment of the present invention. 
     Step S 550 : According to the second comparison result CR 2  of the second comparison module  402 , the synchronization processing module  404  determines whether the storing row number RW of the main image frame  600  that the former stage circuit  120  currently stores exceeds the reading row number RR of the main image frame  600  that the latter stage circuit  130  currently reads. 
     Step S 560 : When the storing row number RW exceeds the reading row number RR, the content read by the latter stage circuit  130  does not exceed the content stored by the former stage circuit  120 . As a result, by delivering a latter stage circuit control signal CC 1 , the synchronization processing module  404  allows the latter stage circuit  130  to keep reading the N-th main image frame and process the N-th main image frame. 
     Step S 570 : On the contrary, when the storing row number RW does not exceed the reading row number RR, the content read by the latter stage circuit  130  exceeds the content stored by the former stage circuit  120 . As a result, by delivering the latter stage circuit control signal CC 1 , the synchronization processing module  404  stops the latter stage circuit  130  from reading the N-th main image frame to avoid the reading of the incorrect data content. 
     As a result, under the coordination and the synchronization of the synchronization circuit  150 , the latter stage circuit  130  can read the content of the N-th main image frame in an order to process the N-th main image frame and generate the main image frame corresponding to the processed main image stream EMS. It is appreciated that in the present embodiment, the latter stage circuit  130  finishes processing the N-th main image frame before the next synchronization signal time period TS N+1 . 
     Furthermore, within the image frame processing time period T N+1 , the former stage circuit  120  can store the N+1-th sub image frames of the sub image streams SS 1  and SS 2  and the N+1-th main image frame of the main image stream MS by using the method described above. 
     Further, by using the method described above, the latter stage circuit  130  can read and process the N-th sub image frames of the sub image streams SS 1  and SS 2  first and subsequently reads and processes the N+1-th main image frame of the main image stream MS. 
     It is appreciated that in an embodiment, the current first sub image storage block  114 A and the second sub image storage block  116 A that the former stage circuit  120  uses to store the N-th sub image frames of the sub image streams SS 1  and SS 2  within the image frame processing time period T N  become the previous sub image storage blocks in the image frame processing time period T N+1 . The previous first sub image storage block  114 B and the previous second sub image storage block  116 B in the image frame processing time period T N  becomes current sub image storage block in the image frame processing time period T N+1  and are used to store the N+1-th sub image frames of the sub image streams SS 1  and SS 2 . 
     The embodiments described above use the condition that the capacity of the main image storage block  112  is larger than or equals to the size of the N-th main image frame as an example. In another embodiment, the capacity of the main image storage block  112  can be smaller than the size of the N-th main image frame and the main image storage block  112  operates as a ring buffer. In other words, when the part of the main image frame stored by the former stage circuit  120  reaches the highest address of the main image storage block  112 , the latest data of the image frame further replaces the lowest address of the main image storage block  112 . When the part of the main image frame read by the latter stage circuit  130  reaches the highest address of the main image storage block  112 , the data is kept reading from the lowest address of the main image storage block  112 . 
     Reference is now made to  FIG. 7  and  FIG. 8  at the same time.  FIG. 7  is a block diagram of the synchronization circuit  150  in another embodiment of the present invention.  FIG. 8  is a flow chart of a synchronization flow  800  performed during the operation of the synchronization circuit  150  in another embodiment of the present invention. The embodiment of the main image storage block  112  that operates as the ring buffer is further described in detail in accompany with  FIG. 7  and  FIG. 8  in the following paragraphs. 
     The synchronization circuit  150  illustrated in  FIG. 7  is similar to the synchronization circuit  150  illustrated in  FIG. 4  and includes the first comparison module  400 , the second comparison module  402  and the synchronization processing module  404 . In the present embodiment, the synchronization circuit  150  illustrated in  FIG. 7  further includes a first calculation module  406 . The first calculation module  406  is configured to calculate a difference DR between the storing row number RW and the reading row number RR according to the frame processing information FP 1  and FP 2 . 
     Besides the flow including the steps S 510  to S 570  illustrated in  FIG. 5 , the synchronization mechanism performed by the synchronization circuit  150  illustrated in  FIG. 7  further includes the flow illustrated in  FIG. 8 . 
     Step S 810 : The first calculation module  406  calculates the difference DR between the storing row number RW and the reading row number RR. 
     Step S 820 : The synchronization processing module  404  determines whether the difference DR exceeds the capacity of the main image storage block  112 . 
     Step S 830 : When the difference DR does not exceed the capacity of the main image storage block  112 , the internal effective content that is not encoded does not fully occupy the capacity of the main image storage block  112 . As a result, by using the former stage circuit control signal CC 2 , the synchronization processing module  404  allows the former stage circuit  120  to keep storing the N-th main image frame. 
     Step S 840 : On the contrary, when the difference DR exceeds the capacity of the main image storage block  112 , the speed that the former stage circuit  120  stores the main image frame is faster than the speed that the latter stage circuit  130  reads the main image frame. If the storing is kept being performed, the data of the main image frame that the latter stage circuit  130  has not read yet is going to be replaced. As a result, by using the former stage circuit control signal CC 2 , the synchronization processing module  404  stops the former stage circuit  120  from storing the N-th main image frame. The storing of the data of the main image frame is therefore stopped temporarily. 
     As a result, by using the mechanism described above, the embodiment of the main image storage block  112  operating as a ring buffer not only avoids the condition that the incorrect data is read when the speed of the read operation of the latter stage circuit  130  is too fast, but also avoids the condition that the data that the latter stage circuit  130  has not read is replaced when the speed of the storage operation of the former stage circuit  120  is too fast. 
     Reference is now made to  FIG. 9  at the same time.  FIG. 9  is a timing diagram of the image processing performed by the multi-stream image processing apparatus  100  in another embodiment of the present invention. 
     The time sequence during the processing of the multi-stream image processing apparatus  100  includes a plurality of image frame processing time periods, such as the consecutive three image frame processing time periods T N−1 , T N  and T N+1  exemplarily illustrated in  FIG. 9 . Each two adjacent image frame processing time periods include a synchronization signal time period therebetween, such as the synchronization signal time period TS N  between the image frame processing time periods T N−1  and T N , and the synchronization signal time period TS N+1  between the image frame processing time periods T N  and T N+1 . The synchronization signal time periods TS N  and TS N+1  respectively correspond to the transmission of the synchronization signals S N  and S N+1  to respectively mark the beginning of the image frame processing time periods T N  and T N+1 . 
     In the present embodiment, during the first sub period T S1  in the image frame processing time period T N , the latter stage circuit  130  still reads and processes the N−1-th sub image frames of the sub image streams SS 1  and SS 2 . During the second sub period T S2  in the image frame processing time period T N , the latter stage circuit  130  still reads and processes the N-th main image frame. 
     However, in the present embodiment, besides the feature that the main image storage block  112  is implemented by a ring buffer, the image frame processing time period T N  further includes a third sub period T S3  before the first sub period T S1 . For the N−1-th main image frame that the image frame processing time period T N−1  corresponds to, the latter stage circuit  130  not only processes the N−1-th main image frame in a latter part of the image frame processing time period T N−1 , but also processes the N−1-th main image frame in the synchronization signal time period TS N  and the third sub period T S3 . Identically, the latter stage circuit  130  can also process the N-th main image frame in the latter part of the image frame processing time period T N , the synchronization signal time period TS N+1  and the third sub period T S3  in the image frame processing time period T N+1 . 
     However, for the former stage circuit  120 , the N-th sub image frames of the sub image streams SS 1  and SS 2  and the N-th main image frame of the main image stream MS are still stored from the beginning of the image frame processing time period T N . As a result, within the third sub period T S3 , the N−1-th main image frame is read and the N-th main image frame is stored simultaneously in the main image storage block  112 . The advantage of applying the third sub period T S3  is that the latter stage circuit can perform data processing with a slower speed and accomplish a power-saving mechanism. 
     Reference is now made to  FIG. 10  and  FIG. 11  at the same time.  FIG. 10  is a block diagram of the synchronization circuit  150  in yet another embodiment of the present invention.  FIG. 11  is a flow chart of a synchronization flow  1100  performed during the operation of the synchronization circuit  150  in yet another embodiment of the present invention. The embodiment of the main image storage block  112  that operates as the ring buffer is further described in detail in accompany with  FIG. 10  and  FIG. 11  in the following paragraphs. 
     The synchronization circuit  150  illustrated in  FIG. 10  is similar to the synchronization circuit  150  illustrated in  FIG. 7  and includes the first comparison module  400 , the second comparison module  402 , the synchronization processing module  404  and the first calculation module  406 . In the present embodiment, the synchronization circuit  150  illustrated in  FIG. 10  further includes a second calculation module  408 . The second calculation module  408  is configured to operate when the image frame that the former stage circuit  120  currently stores and the image frame that the latter stage circuit  130  currently reads are different. 
     The synchronization mechanism performed by the synchronization circuit  150  illustrated in  FIG. 10  includes the flow including the steps S 510 , S 520  and S 540  to S 570  illustrated in  FIG. 5  and the flow including the steps S 810  to S 840  illustrated in  FIG. 8 . However, in the present embodiment, when the image frame that the former stage circuit  120  currently stores and the image frame that the latter stage circuit  130  currently reads are determined to be different in the step S 520  in  FIG. 5 , e.g. the condition illustrated in the third sub period T S3 , the synchronization circuit  150  illustrated in  FIG. 10  operates the flow illustrated in  FIG. 11  to activate the synchronization mechanism. 
     Step S 1110 : According to the frame processing information FP 1  and FP 2 , the second calculation module  408  calculates a difference between the size FR of the N−1-th main image frame and the reading row number RR of the N−1-th main image frame. Such a difference stands for the remained data amount in the N−1-th main image frame that is not read and processed by the latter stage circuit  130 . Subsequently, the second calculation module  408  calculates a sum SUM of the difference and the storing row number RW of the N-th main image frame. The storing row number RW stands for the data amount in the N-th main image frame that the former stage circuit  120  is stored. 
     Step S 1120 : the synchronization processing module  404  determines whether the sum SUM exceeds the capacity of the main image storage block  112 . 
     Step S 1130 : When the sum SUM does not exceed the capacity of the main image storage block  112 , the speed that the former stage circuit  120  performs storage operation does not exceed the speed that the latter stage circuit  130  performs read operation. As a result, by using the former stage circuit control signal CC 2 , the synchronization processing module  404  allows the former stage circuit  120  to keep storing the N-th main image frame. 
     Step S 1140 : On the contrary, when the sum SUM exceeds the capacity of the main image storage block  112 , the speed that the former stage circuit  120  performs storage operation exceeds the speed that the latter stage circuit  130  performs read operation. If the storing is kept being performed, the data of the main image frame that the latter stage circuit  130  has not read yet is going to be replaced. As a result, by using the former stage circuit control signal CC 2 , the synchronization processing module  404  stops the former stage circuit  120  from storing the N-th main image frame. The storing of the data of the main image frame is therefore stopped temporarily. 
     As a result, the multi-stream image processing apparatus  100  of the present invention can use a single latter stage circuit to process the plurality of image streams generated by the former stage circuit with a time-division method to avoid the cost of disposition of a plurality of latter stage circuits. 
     Further, by using the time-division method, the memory module only needs to store N−1-th sub image frame and the N-th sub image frame corresponding to the sub image frames having a lower data amount due to the lower resolution. For the main image frame having a higher data amount due to the higher resolution, the disposition of two storage blocks is not necessary. The space cost of the memory module can be reduced as well. 
     Furthermore, by disposing the synchronization circuit, the memory module can include the main image storage block implemented by the ring buffer. The capacity can be reduced to further reduce the space cost of the memory module. 
     It is appreciated that the embodiments described above are merely an example. In other embodiments, it should be appreciated that many modifications and changes may be made by those of ordinary skill in the art without departing, from the spirit of the invention. 
     In summary, the multi-stream image processing apparatus and the multi-stream image processing method of the present invention can reduce the hardware disposition cost of the latter stage circuit and the space cost of the memory. 
     The aforementioned descriptions represent merely the preferred embodiments of the present disclosure, without any intention to limit the scope of the present disclosure thereto. Various equivalent changes, alterations, or modifications based on the claims of present disclosure are all consequently viewed as being embraced by the scope of the present disclosure.