Abstract:
A multi-processing unit reduces the time to compute parameters of a digital image to by the number of computers operating in the parallel. An image of a digital picture taken during a test of a digital camera module is portioned into N independent portions and each portion is stored into one of N memories. N processors compute test parameters of the image, where each processor works independently on a portion of the image and in parallel with the other processors. The serial computational content of the image is zero allowing a speed-up of the multiprocessing unit to be N with respect to the running the entire computation on a single processor.

Description:
RELATED PATENT APPLICATION  
       [0001]     This application is related to U.S. patent application docket number DS04-022, Ser. No. ______, filed on ______, and assigned to the same assignee as the present invention.  
         [0002]     This application is related to U.S. patent application docket number DS04-023, Ser. No. ______, filed on ______, and assigned to the same assignee as the present invention.  
         [0003]     This application is related to U.S. patent application docket number DS04-024, Ser. No. ______, filed on ______, and assigned to the same assignee as the present invention.  
         [0004]     This application is related to U.S. patent application docket number DS04-026, Ser. No. ______, filed on ______, and assigned to the same assignee as the present invention.  
         [0005]     This application is related to U.S. patent application docket number DS04-027, Ser. No. ______, filed on ______, and assigned to the same assignee as the present invention. 
     
    
     BACKGROUND OF THE INVENTION  
       [0006]     1. Field of Invention  
         [0007]     The present invention is related to computer multi-processing and in particular to parallel processing of picture data captured during a test of a digital camera module.  
         [0008]     2. Description of Related Art  
         [0009]     The digital camera is becoming a ubiquitous device. Not only are digital cameras replacing the traditional film camera, digital camera devices are being used in many other applications, such as small electronic devices, such as PDA (personal data assistant) and cellular phones. With the explosion of cellular phones, the ability to take a picture and then send that picture to another individual using a second cellular phone comes the need to produce inexpensive digital camera modules and efficiently test these modules in large quantities. This is further complicated by the many different module configurations that are emerging as a result of the many different application requirements, including fixed focus, manual focus and automatic focus as well as physical size. Some of these modules are very small and others have signal leads in the form of a flex filmstrip. The testing time for digital camera module, which can have mega-pixel capability, has traditionally been a relatively long process (approximately sixty seconds for a module with 0.3 megapixels) to insure the integrity and picture quality of the camera. Many computations performed on picture images are necessary to insure that the pixel elements of the digital camera module meet specifications. Quality testing at a low cost has become the utmost of importance. This necessitates a testing capability that is fast (including the computations that must be made) and insures the integrity and specification of the digital camera module while testing a large quantity of modules.  
         [0010]     A patent application, Ser. No. 10/417,316 dated Apr. 16, 2003, is related to miniature cameras and their manufacturing methods that are used as built-in modules in hand held consumer electronics devices such as mobile phones and PDA&#39;s. In a second patent application, Ser. No. 10/434,743 dated May 18, 2003, a test system is described for digital camera modules used as built-in modules for consumer electronics, which performs electrical tests, adjustment of focus and sealing of the lens barrel with glue.  
         [0011]     In addition there are a number of other prior art patents that are directed to testing of digital cameras: US 20040032496A1 (Eberstein et al.) is directed to a method of camera calibration and quality testing; EP 1389878A1 (Bednarz et al.) is directed to a method of camera calibration and testing camera quality; US 20040027456A1 (pierce) directed to the use of calibration targets; EP 1382194A1 (Baer) is directed to dark current subtraction; JP 2003259126 (Keisuke) is directed to removing noise of an image; US 20030146976A1 (Liu) is directed to a digital camera system enabling remote monitoring; JP 2003219436 (Fuminori) is directed to adjustment of a pixel shift camera; US 2003142374 (Silverstein) is directed to calibrating output of an image output device; JP 2003179949 (Hidetoshi) is directed to a luminance level inspection apparatus; JP 2003157425 (Vehvilainen) is directed to improving image quality produced in a mobile imaging phone; JP 2003101823 (Kenichi) is directed to specifying a picture data area; EP 1286553 A2 (Baer) is directed to a method and apparatus for improving image quality; US 20030030648 (Baer) is directed to a method and apparatus for improving image quality in digital cameras; U.S. Pat. No. 6,512,587 (Dilella et al.) is directed to measurement method and apparatus of an imager assembly; US 20030002749 (Vehvilainen) is directed to a method and apparatus for improving image quality; US 20020191973 A1 (Hofer et al.) is directed to a method and apparatus for focus error reduction; WO 2002102060 A1 (Baer) is directed to a method and apparatus for smear in digital images using a frame transfer sensor; JP 2002290994 (Hidetoshi) is directed to a method and apparatus to detect foreign matter on the surface of a lens; JP 200223918 (Yanshinao) is directed to an image inspection device and method for a camera module; JP 2002077955 (Keisuke) is directed to a method and apparatus for evaluating camera characteristics; JP 2001292461 (Keisuke) is directed to a system and method for evaluating a camera; U.S. Pat. No. 6,219,443 B1 (Lawrence) is directed to a method and apparatus for inspecting a display using a low resolution camera; U.S. Pat. No. 6,201,600B1 (Sites et al.) is directed to a method and apparatus for inspection of optically transmissive objects having a lens; U.S. Pat. No. 5,649,258 (Bergstresser et al.) is directed to an apparatus and testing of a camera; EP 0679932 B1 (Kobayashi et al.) is directed to testing an electronically controlled camera; U.S. Pat. No. 5,179,437 (Katsumi et al.) is directed to an apparatus for color correction of image signals of a color television camera; JP 03099376 (Hiroshi) is directed to the quality of a display screen; U.S. Pat. No. 4,612,666 (King) is directed to a pattern recognition apparatus; and U.S. Pat. No. 4,298,944 Stoub et al.) is directed to a method and apparatus for distortion correction for scintillation cameras  
       SUMMARY OF THE INVENTION  
       [0012]     It is an objective of the present invention to provide a plurality of parallel processors, which operate separately on separate portions of a digital picture image taken by a digital camera module under test.  
         [0013]     It is further an objective of the present invention to compute parameters of a digital picture image in a separate portion of the image independent of other portions of the image.  
         [0014]     It is still further an objective of the present invention to speed up the computation time of a digital picture image by approximately the number of independent computer processors each operating on an independent portion of the digital picture image.  
         [0015]     It is also further an objective of the present invention to synchronize a light source with a clock of a digital camera module under test to capture a picture image, separate the picture image into independent portions and store the independent portions into independent memories to be operated on by independent computer processors.  
         [0016]     In the present invention a computational unit comprises N memories and N computer processors units (CPU), where N is an integer. The N memories and N CPU form a computational unit to compute image parameters of a test picture image taken by a digital camera module under test (MUT). A frame grabber synchronizes the clock of MUT with a fast light source to capture a picture image. The light source ranges in time from approximately one microsecond to one hundred milliseconds. The picture image is partitioned into N portions and coupled to the N memories by a frame grabber, which synchronizes the coupling of the N portions into the N memories. A first picture portion of the N picture portions is coupled to a first memory. A second picture portion of the N picture portions is coupled to a second memory. An Nth picture portion of the N picture portions is coupled to an N memory.  
         [0017]     A first memory of the N memories is paired with a first CPU of the N CPU to operate on a first picture portion of N picture portions independent of the remaining N−1 picture portions. A second memory of the N memories is paired with a second of the N CPU to operate on a second picture portion of N picture portions independent of the remaining N−1 picture portions. An Nth memory of the N memory portions is paired with an Nth CPU of the N CPU to operate on an Nth picture portion of the N picture portions independent of the remaining N−1 picture portions. The N CPU, operating independently of the each other computes the image parameters of each of the N pictures portions independently of the other picture portions.  
         [0018]     The time to process a job in a single processor is t=ts+tp, where ts=the time to process the serial content of a job and tp=the time to process the parallel content in a single processor. From parallel processing technology it is well known that the time to process a job in a parallel processor is t=ts+tp/N, where N=the number of parallel processors. Since the N picture portions are loaded into N separate memories in which the picture portions and the memories are independent of each other and since the N memory and N CPU combinations operate independent of each other, then ts=0, and the speed-up (SU) in the computation time of the N picture portions is SU=(ts+tp)/(ts+tp/N)=N. Therefore, the processing of the N picture portions by the N processors provides a reduction in the computational time by N, the number of processors that are operating simultaneously on the separate picture portions.  
         [0019]     A plurality of frame grabbers can be used in which portions of a plurality of picture images are split and processed by a separate set of computer processing units. This separate set of computer processing units can comprise small computing units, i.e. PC&#39;s (personal computers), and/or additional NCPU. The picture image can be coupled into the memory by pixel allowing computation on pixels coupled to the memory while additional pixels of the picture image are being loaded in a video streaming fashion. The video streaming provides a capability to overlap the computation with the loading of the memory. A plurality of frame grabbers can be used to couple the video streaming of the picture image into the NCPU in a fashion where the plurality of frame grabbers and NCPU form a pipeline structure. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0020]     This invention will be described with reference to the accompanying drawings, wherein:  
         [0021]      FIG. 1  is a block diagram of the parallel processing unit of the present invention,  
         [0022]      FIG. 2  is a block diagram of the interaction of a frame grabber with the parallel processor memory, the light source and the digital camera module of the present invention,  
         [0023]      FIG. 3A through 3C  are diagrams of a portioned digital picture image of the present invention,  
         [0024]      FIG. 4  is a block diagram of the present invention showing the storing of picture portions by the frame grabber, and  
         [0025]      FIG. 5  is a flow diagram of the present invention for synchronizing the taking of a digital picture and storing picture portions into N independent memories. 
     
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT  
       [0026]     In  FIG. 1  is shown a computational unit  13  of the present invention for use in computing picture parameters of digital picture images captured by the digital camera module (MUT)  10 . Image data  24  of a digital picture is coupled from the MUT  10  to the memories  14 ,  16 ,  18 , and  20  of the computational unit  13  through pin electronics  11 , connected to the signal pins of the MUT, and a test interface  12 . The select  25  selects the memory into which image data  24  is to be stored. The sync out  23  of each memory  14 ,  16 ,  18  and  20  allows the image data  24  to be portioned between the memories, preferably in equal data amounts, by allowing each memory to be synchronized with a portion of the image data.  
         [0027]     Each memory  14 ,  16 ,  18  and  20  has a field programmable gate array (FPGA) that controls the loading of the image data  24  into the memory and delivery of computed results by a link  22  to a host computer. Memory (1)  14  is coupled to a CPU (1)  15  and acts as a computational pair to operate on a first portion of a picture image stored in memory (1)  14 . In like manner memory (2)  16  is coupled to CPU (2)  17  to operate on a second portion of the picture image stored in memory (2), memory (3)  18  is coupled to CPU (3)  19  to operate on a third portion of the picture image stored in memory (3), and memory (4)  20  is coupled to CPU (4)  21  to operate on a fourth portion of the picture image stored in memory (4).  
         [0028]     Each memory  14 ,  16 ,  18  and  20  is loaded with a different portion of a picture image for computation by the respective CPU  15 ,  17 ,  19 , and  21 . Alternatively, each memory  14 ,  16 ,  18 ,  20  is loaded with a different picture using a different frame grabber such that a first frame grabber loads a first picture image into a first memory  14 , a second frame grabber loads a second picture image into a second memory  16 , a third frame grabber loads a third picture image into a third memory  18  and a fourth frame grabber loads a fourth picture image into a fourth memory  20 . In addition the first, second, third and fourth picture images can be streamed into the first, second, third and fourth memories  14 ,  16 ,  18  and  20  by pixel in a pipeline fashion.  
         [0029]     Continuing to refer to  FIG. 1 , there is no data that is transferred between the memories  14 ,  16 ,  18  and  20 , or between the CPU&#39;s  15 ,  17 ,  19  and  20 ; therefore, the respective pairs of memory and CPU  14  and  15 ,  16  and  17 ,  18  and  19 , and  20  and  21  operate in parallel independent of each other. This allows the computational time on a picture image in the computation unit  13  to be performed in 1/N=1/4 of the time that it takes to process the picture image data using a single processor, where N=the number of memory and CPU pairs. There is no fundamental limit to the number of memory and CPU pairs and a number of processors greater than 4 is within the scope of the present invention. For a number N&gt;4 of memory and CPU pairs to be effective, the picture image captured by the MUT  10  needs to be portioned into N equal portions that can be run simultaneous in the N&gt;4 memory and CPU pairs. It should be noted that fewer than four processors can be used, and the processors can operate on more than one portion.  
         [0030]     In  FIG. 2  is a block diagram showing the synchronization by a frame grabber  30  for the capturing of a digital image and storing of the image in the memory of a computational unit  13  ( FIG. 1 ). The frame grabber  30  first synchronizes  31  a light source  32 , which produces a selected light  34  that is focused on the MUT  10 , with a clock  35  of the MUT  10  to capture a digital picture image. The resulting image data  24  is then stored in the memory of the computational unit  13  by the frame grabber  30 . The frame grabber using memory select  25  selects the computational memory  14 ,  16   18 , and  20  ( FIG. 1 ) and synchronizes the image data  24  with the sync out signal  23  ( FIG. 1 ) of each memory to load a portion of the image data  24  into each of the computational memories  14 ,  16 ,  18 , and  20  ( FIG. 1 ) such that each memory  14 ,  16 ,  18 ,  20  contains a different portion of the image data.  
         [0031]     In  FIG. 3A  is shown a preferred digital picture image  40   a  containing pixels  41  in rows and columns in which the image is partitioned into portions (1a)  42 , (2a)  43 , (3a)  44  and (4a)  45  that occupy the corner quadrants of the image  40   a.  A second partition of a digital picture image  40   b  is shown in  FIG. 3B  in which the portions (1b)  46 , (2b)  47 , (3b)  48  and (4b)  49  are divided into rows of pixels  41 . A third partition of a digital picture image  40   c  is shown in  FIG. 3C  in which the portions (1c)  50 , (2c)  51 , (3c)  52  and (4c)  53  are divided into columns of pixels  41 . It is within the scope of the present invention that other partitions to can create portions of the image data  24  comprising diagonal partitions and partitions of concentric circles that are used to provide image data to the computational unit  13  ( FIG. 1 ). Also the number of partitions, which create portions, is directly related to the number of CPU  15 ,  17 ,  19  and  21  ( FIG. 1 ). If there are additional CPU (CPU&gt;4) in the computational unit than shown in  FIG. 1 , the number of portions will increase to equal the number of CPU. Similar, if there are fewer CPU (CPU&lt;4) in the number of portions will decrease to equal the number of CPU. It is also within the scope of the present invention that each CPU  15 ,  17 ,  19  and  21  ( FIG. 1 ) operates on more than one portion.  
         [0032]     In  FIG. 4  is a diagram exemplifying the coupling by the frame grabber of portions  42 ,  43 ,  44 , and  45  of a digital picture image  40   a  to the memories  14 ,  16 ,  18  and  20  of the computational unit  13 . The digital picture image  40   a  comprises rows of pixels  41  from 1≦R≦2M and columns of pixels  41  from 1≦C≦2K, where R is a row of pixels and C is a column of pixels. The frame grabber  30  scanning rows of the digital picture image from C=1 to C=2K selects memory (1)  14  and stores a first row R=1 of pixel  41  data from 1≦C≦K within quadrant (1)  42  into memory (1)  14  by synchronizing the image data  24  within the quadrant (1) with the sync out  23  of memory (1)  14 . Then for columns of pixels K+1≦C≦2K in quadrant (2)  43 , the frame grabber stores the remainder of the first row R=1 of pixel  41  data into memory (2)  16  using the select signal  25  and sync out signal  23  of the memory (2)  16 . Then the first part of the next row R=2 of pixel  41  data for columns 1≦C≦K is stored into memory (1) using the select  25  and sync out  23  signals connected to memory (1)  14 . At the boundary between portion (1) and portion (2), the frame grabber selects  25  memory (2)  16  and stores the remainder of row R=2 of pixel  41  data for columns K+1≦C≦2K into memory (2)  16  using the sync out  23  of memory (2). This alternate storing of a first part of a row R=M for columns 1≦C≦K of pixel  41  data first into memory (1)  14  and then the second part of the row R=M of pixel  41  data for columns K+1≦C≦2K into memory (2)  16  continues until all of the pixel data in quadrant (1)  42  and quadrant (2)  43  have been stored. Then the frame grabber stores rows M+1≦R≦2M of pixel  41  data into Memory (3)  44  and memory (4)  45  in a similar fashion.  
         [0033]     Continuing to refer to  FIG. 4 , the frame grabber  30  selects memory (3)  18  upon completion of the storing of pixel data into memory (1)  14  and memory (2)  16  and stores a row R=M+1 of pixel  41  data for columns 1≦C≦K within quadrant (3)  44  into memory (3)  18  by synchronizing the data within the quadrant (3) with the sync out  23  of memory (3)  18 . At the boundary between quadrant (3)  44  and quadrant (4)  45 , the frame grabber stores the remainder of the row R=M+1 of pixel  41  data for columns K+1≦C≦2K into memory (4)  20  using the select  25  and sync out  23  signals of the memory (4)  20 . Then the first part of the row R=M+2 of pixel  41  data for columns 1≦C≦K is stored into memory (3)  18  using the select  25  and sync out  23  signals connected to memory (3). At the boundary between portion (3)  44  and portion (4)  45 , the frame grabber  30  selects  25  memory (4)  20  and stores the remainder of the row R=M+2 of pixel  41  data for columns K+1≦C≦2K into memory (4)  20  using the sync out  23  of memory (4). This alternate storing of a first part of a row M+1≧R≧2M of pixel  41  data for columns 1≦C≦K first into memory (3)  18  and then the second part of the row M+1≧R≧2M of pixel  41  data for columns K+1≦C≦2K into memory (4)  20  continues until all of the pixel data in quadrant (3)  44  and quadrant (4)  45  has been stored. Upon completion of the storing of the image data  24  into memories  14 ,  16 ,  18  and  20 , the parallel processors  15 ,  17 ,  19  and  20  operate in parallel on the data independently of each other to compute image parameters of the digital picture image  40   a,  which reduces the computation time by 1/N=1/4 for the computational unit  13  shown in  FIG. 1  as compared to the time required to perform the computation by a single processor. It should be noted that the scan of the pixel data can be scanned in the reverse direction for each row by column from C=2k to C=2K for columns 2K≧C≧1 where the image data from 2K≧C≧2K+1 is stored into memory (2)  16 , then the image data for columns K≧C≧1 is stored into memory (1)  14  for rows 1≦R≦M, and then image data for columns 2K≧C≧K+1 is stored into memory (4)  20  and image data from K≧C≧1 is stored into memory (3)  18  for rows M+1≦R≦2M.  
         [0034]     Continuing to refer to  FIG. 4 , an alternative method for loading the memories  14 ,  16 ,  18  and  20  with image data  24  is to store all image data for a first memory before storing any image data for a second memory. This is done by scanning out of the MUT  10  ( FIG. 2 ) image data containing pixel  41  data by rows 1≦R≦M for columns 1≦C≦K and storing the image data  24  into memory (1)  14  using the frame grabber to select  25  memory (1) and synchronize the pixel data with the memory (1) using sync out  23 . Then scan out of the MUT  10  image data containing pixels  41  by rows 1≦R≦M for columns K+1≦C≦2K and storing the pixel data into memory (2)  16  using the frame grabber to select  25  memory (2) and synchronize the image data with the memory (2) using sync out  23 . Next, scanned out of the MUT  10  ( FIG. 2 ) image data containing pixel  41  data by rows M+1≦R≦2M for columns 1≦C≦K and storing the pixel data into memory (3)  18  using the frame grabber to select  25  memory (3) and synchronize the image data with the memory (3) using sync out  23 . Then scan out of the MUT  10  image data  24  containing pixel  41  data by row M+1≦R≦2M for columns K+1≦C≦2K and storing the image data into memory (4)  20  using the frame grabber to select  25  memory (4) and synchronize the image data with the memory (4) using sync out  23 . It should be noted that the image data can be scanned out in reverse order by row for columns from C=2K to C=1. Also the image data can be scan out in reverse order of rows from R=2M to R=1 for columns in either order C=1 to C=2K or C=2K to C=1.  
         [0035]     A flow diagram in  FIG. 5  demonstrates the method to capture an image from a digital camera module (MUT)  10  ( FIG. 1 ) and couple that image to a computer processing unit  13  for computing the image parameters in a plurality of image portions where each portion is computed separately by a processor operating in parallel with other processors operating separately on the image portions. A light color in a light source is selected  60 . A frame grabber synchronizes the light source having a fast turn-on and turn-off in the order of microseconds with the clock of the MUT  61  so that the time occurrence of the light and the time occurrence of the capture of the image  62  of the light are concurrent. The frame grabber divides the image into N portions  63 , where N is an integer. Pixel data of a first image portion of the N image portions is synchronized with a first memory of N separate memories in a computational unit  64  by the frame grabber, and then the pixel data of the first image portion is stored into a first memory of N memories  65 . Then pixel data of a second image portion of the N image portions is synchronized with a second memory of N separate memories in a computational unit  64  by the frame grabber, and then the pixel data of the second image portion is stored into a second memory of N memories  65 . If N is not the last N  66 , the process is repeated until all pixel data of the N portions are stored into the N memories  67 , each time using the frame grabber to synchronize the data with the sync out signal the N memory. After all pixel data of the image portions are stored into the N memories, then a first processor of N processors, computes image parameters for a first image portion stored in a first memory of the N memories in parallel with a second processor computing image parameters for a second image data in a second memory in parallel with the remaining N−2 processors computing image parameters separately for the remaining N−2 image data portions located in the remaining N−2 memories  68 .  
         [0036]     While the invention has been particularly shown and described with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made without departing from the spirit and scope of the invention.