Signal processing circuit for a solid-state image pickup device

Abstract: A signal processing circuit of a solid-state imaging device having an image pickup section formed by plural solid-state image sensors (4) including a discrete pixel array is dis­ closed. The solid state image sensor (4G) for green-color image pickup is arrayed in the image pickup section with a spatial shift equal to one half the pixel pitch from the solid-state image sensors (4R, 4B) for red-color image pickup and the solid-state image sensors for blue-color image pickup in accordance with the spatial pixel shifting method. An optical low pass filter (2, 3) having a cut-off frequency equal to fs is provided on the front surfaces of these solid-­ state image sensors (4). The green-color image pickup output signal, the red-color image pickup output signal and the blue­ -color image pickup output signal, produced at the solid-state image sensors (4) of the image pickup section, are subjected to A/D conversion, followed by phase processing with the sampling phases of the respective digital signals matched with one another, to produce high-quality image output signals having satisfactory modulation transfer function (MTF) charac­ teristics and containing lesser amounts of aliasing com­ ponents.

Description: This invention relates to a signal processing circuit of a solid-state image pickup device an image pickup section of which includes a plurality of solid state image sensors each having a discrete pixel array and a camera including such a signal processing circuit. In a solid-state image pickup device the image pickup section of which includes solid-state image sensors each having a discrete pixel array formed by charge coupled devices (CCDs), aliasing components from the spatial sampling frequency fs are admixed into the image pickup output signals of the solid-state image sensors, as shown by hatched lines in Fig. 1. This is because the solid-state image sensors are of the sampling system. The conventional practice to prevent the aliasing component from being produced into the base band of the image pickup output signal is to provide a double refraction type optical lowpass filter in the image pickup optical system to suppress the high frequency part of the baseband component of the image pickup signal to meet the Nyquest conditions of the sampling system by the solid state image sensors. In the field of color video cameras, there are placed on the market a multi-plate type solid state image pickup device, such as a two-plate type solid state image pickup device in which three primary color images are picked up by a solid-state image sensor provided with color coding filters for red-color and blue-color pixels and a solid-state image sensor for green-color image pickup, or a three-plate type solid-state image pickup device for picking up of the three primary color images by separate color solid-state image sensors. As a technique of improving the resolution in the multi-plate type solid-state image pickup device, there is known a so-called spatial pixel shifting method, according to which the cut-off frequency of the optical low-pass filter is selected to be fs and, as shown in Fig. 3, the solid-state image sensors for red color image pickup and blue color image pickup are placed with a shift equal to one half the spatial pixel sampling period with respect to the solid-state image sensor for green color image pickup. By taking advantage of the spatial pixel shifting method, it becomes possible with the multi-plate type analog output solid-state image pickup device to realize high resolution exceeding the limit imposed by the number of pixels of the solid state image sensor. In service digital video tape recorders, employed in, for example, a broadcasting station, standardization of so-called D1/D2 formats are under way and digital interfaces for digital video related equipment conforming to these standards are required to be used in color television camera devices. According to the digital interface standards for the digital video related equipment, the sampling rate is set so as to be approximately equal to the sampling rate fs of the currently employed solid state image sensors. In the solid state image pickup device the image pickup section of which includes solid state image sensors each having a discrete pixel array, such as the aforementioned CCD image sensors, a double refraction type optical low pass filter is provided in the image pickup optical system to prevent the occurence of the aliasing component into the baseband of the image pickup signal by the solid state image sensors. However, since the optical low pass filter cannot be made to have steep cut-off characteristics, the problem is presented that the modulation transfer function or MTF in the high frequency range is liable to be deteriorated in the high frequency range, as shown in Fig. 2. Hence, in the multi-plate type solid-state image pickup device for color image pickup, signal deterioration caused by the aliasing component or MTF occurs unavoidably if ordinary sampling is employed. Meanwhile, when a digital interface for a digital video related equipment is loaded on a solid state imaging device, the image pickup section of which includes solid state image sensors each having a discrete pixel array, such as the CCD image sensors, the digital rate is too high with the sampling rate of the digital interface of 2fs, while the conformity to the digital interface standards cannot be achieved. On the other hand, if the rate of the digital processing system conforming to the D1/D2 format is adopted, the resolution becomes lower than that of the conventional solid state image pickup device for analog application when directly using the analog output. From GB-A-2 114 854 a solid state camera is known comprising analog-to-digital converting means for digitizing image pick-up output signals read out at a sampling rate fs from a solid state image sensor and interpolating filter means for performing sampling phase conversion for making coincident the sampling of the image pick-up output signals digitized by said analog-to-digital converting means and having a clock-rate twice the clock-rate of the sampling rate of the solid state sensors. Therefore, image pick-up output signals are formed having a coincident sampling phase at a clock-rate twice that of said analog-to-digital converting means. From 'Proceedings of the 1985 International Symposium on Circuits and Systems', Kyoto, 5-7/6/1985 pages 1343-1346, New York, US; L. Paris et al.: "An Oversampling Digital Filter for Videocommunications" a digital filter performing interpolation on luminance and chrominance video signals with filters up to 19th order is known. It is an object of the present invention to provide a signal processing circuit in the multi-plate type solid state color image pickup device allowing to perform high-quality image pickup of a solid-state image pickup device including analog-to-digital converting means for digitizing image pickup output signals at a clock rate equal to a sampling rate fs, said image pickup output signals being read at the sampling rate fs from the solid-state image sensor for green-color image pickup and the solid-state image sensors for red-color and blue-color image pickup of the image pickup section, that are arrayed with a spatial shifting equal to one half the pixel pitch from each other; a processing circuit for processing the digital output signals of the analog-to-digital converting means, and color encoder means supplied with output signals of said processing circuit output signal having superior MTF characteristics and containing lesser amounts of aliasing components. It is another object of the present invention to provide a color image pickup device in which a digital interface conforming to the standards for the digital interface related equipment is loaded on the solid state image pickup device the image pickup section of which includes solid state image sensors, each having a discrete pixel array, such as CCD image sensors, to allow to produce high-quality digital composite video signals having satisfactory MTF characteristics and containing lesser amounts of aliasing components. It is a further object of the present invention to provide a signal processing circuit and producing digital composite video signals; whereby the high-quality digital composite video signals having superior MTF characteristics and containing lesser amounts of aliasing components may be produced. It is a further object of the present invention to provide a signal processing circuit of a solid-state image pickup device allowing to produce high-quality digital composite video signals and analog composite video signals having superior MTF characteristics and containing lesser amounts of aliasing components. According to one aspect of the present invention, a signal processing circuit for a solid-state image pickup device as claimed in claim 1 is provided. According to another aspect of the invention a camera is provided including a circuit of the above mentioned type. Fig. 1 is a schematic diagram for illustrating a conventional solid-state image sensor response. Fig. 2 is a schematic diagram for illustrating another conventional solid-state image sensor response. Fig. 3 is a schematic diagram showing the layout of the CCD image sensors according to the pixel shifting method. Fig. 4 is a block diagram showing the signal processing circuit of the soid state image pickup device according to the present invention. Figs. 5A to 5F are schematic frequency diagrams for illustrating the operation of the signal processing circuit shown in Fig. 4. Fig. 6 is similarly a schematic frequency diagram for illustrating the operation of the signal processing circuit shown in Fig. 4. Fig. 7 is a block diagram showing an interpolating section employed in the signal processing section shown in Fig. 4. Fig. 8 is a schematic diagram showing the operation of the interpolating section shown in Fig. 7. By referring to the drawings, a signal processing circuit of a solid-state imaging device according to an embodiment of the present invention will be explained in detail. Fig. 4 is a block diagram showing an example in which the signal processing circuit according to the present invention is applied to a three-plate type solid-state image pickup device. This image pickup device is so arranged and constructed that an imaging light L i incident thereon from an imaging lens 1 by way of an optical low pass filter 2 is divided by a color separation prism 3 into three primary color components to produce three-color images of an object and to form the three-color image by three CCD image sensors 4R, 4G and 4B. These three CCD image sensors 4R, 4G and 4B, making up the image pickup section of the three-plate solid-state image pickup device, are arranged with the use of the spatial pixel shifting method in the manner shown in Fig. 3. Thus the CCD image sensor 4R for forming the red-color image and the CCD image sensor 4B for forming the blue-color image are arranged with a shift equal to one half the spatial pixel sampling period ts with respect to the CCD image sensor 4G for forming the green-color image. The CCD image sensors 4R, 4G and 4B are driven by a CCD driving circuit, not shown, so that image pickup electrical charges of each pixel are read out with read-out clocks having a sampling frequency fs which is four times as high as the color subcarrier frequency fsc, or 4fsc. The three-color images of the object are subjected to spatial sampling at the CCD image sensor 4G for green-color image pickup on the one hand and at the CCD image sensors 4R, 4B for the red-color and blue-color image pickup on the other, at the position spatially shifted by τs/2 from each other. Thus, as may be seen from the spectral components at A in Fig. 5, the sampling frequency component fs of a green-color image pickup output signal S G from the sensor 4G is oppositely phased with respect to the sampling frequency components fs of the red-color and blue-color image pickup output signals S R and S B from the sensors 4R, 4B, respectively. The sampling timing of a red-color image pickup output signal S R by the image sensor 4R and that of a blue-color image pickup output signal S B by the CCD image sensor 4B are shown by a circle ○ at A in Fig. 6, whereas the sampling timing of a green-color image pickup output signal S G by the CCD image sensor 4G is shown by a square □ at B in Fig. 6. The image pickup output signals S R , S G and S B , read out from the CCD image sensors 4R, 4G and 4B with read-out clocks having the aforementioned sampling frequency fs equal to 4fsc, are supplied to A/D converters 6R, 6G and 6B via buffer amplifiers 5R, 5G and 5B, respectively. To the A/D converters 6R, 6G and 6B, there are applied clocks from a timing generator, not shown, at a clock rate equal to the sampling rate of each of the image pickup output signals SR, SG and SB, that is at the clock frequency fs which is equal to 4fsc as are the read-out clocks of the aforementioned CCD image sensors 4R, 4G and 4B. These A/D converters 6R, 6G and 6B directly digitize the image pickup output signals S R , S G and S B , with the clock rate fs equal to 4fsc, to form color data D R , D G and D B having the same output spectra as the spectra of the imaging output signals S R , S G and S B shown at A in Fig. 5. The green color dat D G obtained at the A/D converter 6G for the green-color image pickup output signals are supplied to an interpolating section 7, where they are subjected to interpolation to produce interpolated green color data DG' which are then supplied to an image enhancement section 9 while being simultaneously supplied to a matrix processing section 11. The red color data D R produced from the A/D converter 6R for red-color image forming output signals and the blue color data D B produced from the blue-color image pickup output signals are first delayed by delaying sections 8R, 8G for a delay time equal to the time interval required in performing interpolation at the interpolating section 7, and then supplied to the image enhancement section 9 while being simultaneously supplied via processing sections 10R, 10B to the matrix processing section 11. The green-color data D G digitized from the green-color image pickup output signal S G is subjected to interpolating arithmetic operation at the interpolating section 7, so as to be given the interpolating filtering characteristics having a single zero at fs/2 and a passband in the range from 0 to fs/2, so that the phase correction for shifting the sampling phase of the green-color image pickup output signal S G by Ts/2 is achieved without collective signal delaying. By such phase correcting operation for the green-color image pickup output signal S G , the interpolating section 7 forms green-color interpolation data corresponding to the phase-corrected green-color imaging output signal S G ' and having the same phase as the sampling phase of each of the red-color imaging output signal S R and the blue-color imaging output signal S B . The interpolating section 7 is formed by a digital filter defined by the clock rate fs of the A/D converter 6G for the green color image pickup output signal. That is, as shown in Fig. 7, this interpolating section is made up mainly of 2n delay circuits DL₁ to DL 2n each having a one-clock delay Z⁻¹ in terms of the clock rate fs of the A/D converter 6G, as indicated by Z -1 = 1/fs n adders ADD₁ to ADD n for adding the outputs of the delay circuits DL₁ to DL 2n together, n weighting circuits W₁ to W n supplying weights h₁ to h n to the outputs of the adders ADD₁ to ADD n and an adder ADD₀ supplied with outputs of the adders ADD₁ to ADD n by way of the weighting circuits W₁ to W n , for providing impulse response characteristics H IP (Z) shown in Fig. 8 and represented by H IP (Z) = {(1+Z -1 )/2}H'(Z) The phase-corrected green-color imaging output signal S G ', produced at the interpolating section 7 and indicated as the green-color interpolation data D G ', represents a sampling signal having the frequency fs and in phase with both the red-color imaging output signal S R and the blue-color imaging output signal S B . Thus the green-color image pickup output signal S G ' may be digitally processed, such as by gamma correction, at the clock frequency of fs, by using color data D R , D B indicating the red-color image pickup output signal S R and the blue-color image pickup output S B and the green-color interpolating data D G '. That is, in the present embodiment, in the course of data processing for the green-color interpolation data D G ', red-color data D R and the blue-color data D B , the image enhancement section 9 sums the phase-corrected green-color imaging output signal S G ' on one hand and the red-color imaging output signal S R and the blue-color imaging output signal S B on the other at a ratio of 1:1. In this manner, it is possible to cancel aliasing components having a frequency in the range of 0 to fs/2, as shown at D in Fig. 5. From this summation signal, an image enhancement signal S 1E having the frequency spectrum as shown at E in Fig. 5 is formed. In the processing sections 10R, 10G and 10B, the green-color interpolation data D G ', the red-color data D R and the blue-color data D B are subjected to, above all, gamma correction and profile enhancement by the image enhancement signal S IE . In the matrix processing section 11, data of a luminance signal Y and chrominance signals I and Q are formed on the basis of the green interpolation data D G ' red-color data D R and the blue-color data D B , supplied from the processing sections 10R, 10G and 10B, so as to be then supplied to an encoder 12. From the data of the luminance signal Y and the chrominance signals I and Q 1, the encoder 12 forms and outputs color video data conforming to for example, a so-called D₂ format. It is noted that the luminance signal Y formed by summing the phase-corrected green color imaging output signal S G ' to the red color image pickup output signal S R and the blue color image pickup output signal S B at the predetermined ratio contains a minor amount of the aliasing components in the frequency range of 0 to fs/2, as shown at F in Fig. 5. However, signal deterioration may be reduced by the image enhancement operation by the image enhancement signal S IE that are free from the aliasing components in the frequency range of 0 to fs/2, as mentioned hereinabove. In the above image enhancement section 9, the phase-corrected green-color image pickup output signal S G ' on one hand and the red-color image pickup output signal S R and the blue-color image pickup output signal S B on the other may be summed together at such a level ratio that the aliasing components in the luminance signal Y into the frequency range of 0 to fs/2 may be cancelled, and the image enhancement signal S IE may be formed from this summation signal. In the above embodiment, the green color interpolation data D G ' corresponding to the phase-corrected green-color image pickup output signal S G ' having the same phase as the sampling phase of each of the red-color image pickup output signal S R and the blue-color image pickup output signal S B is formed from the green-color data D G digitized from the green-color image pickup output signal S G by the above described phase correction performed at the interpolating section 7. However, the present invention is not limited to the above embodiment. For example, an interpolating section may be provided for subjecting the red-color data D R digitized from the red-color image pickup signal S R and the blue-color data D B digitized from the blue color image pickup output signal S B to the interpolating similar to the above mentioned phase correction for the green color data D G to form the interpolation data for the respective colors corresponding to the phase-corrected red-color and blue-color image pickup output signals having the same phase as the sampling phase of the green-color image pickup output signal S G . In the above described signal processing circuit of the present invention, the processing system in its entirety may be caused to operate at a lower clock rate fs corresponding to the spatial sampling frequency of each of the CCD image sensors 4R, 4G and 4B making up the image pickup section of the solid-sate image pickup device. The interpolating section 7 performing phase correction performs digital data processing of the color data digitized from the image pickup output signal resulting in extremely satisfactory delay and group delay characteristics. The CCD image sensors 4R, 4G and 4B making up the image pickup section may be of the subnyquist system to increase the MTF in the frequency range of 0 to fs/2. Moreover, the aliasing components into The above mentioned frequency range from 0 to fs/2 may be cancelled to produce the imaging output signal of extremely high image quality by the processing corresponding to the spatial pixel shifting method. In the signal processing circuit of the present invention, the imaging output signals read out at the sampling rate fs from the solid state image sensors for green color, red color and blue color imaging of the image pickup section, in which the above three solid state image sensors are placed with spatial shifting by one half the repetitive pitch of the pixels, are digitized by analog-to-digital converting means at the clock rate equal to the sampling rate fs. The green color picture imaging output signal or the red color picture image pickup output signal and the blue color picture image pickup output signal, thus digitized by the analog-to-digital converting means, are subjected by interpolating filter means to a sampling phase conversion having a single zero at fs/2 and exhibiting filter characteristics having a passband in the vicinity of 0 to fs/2 to produce a green color picture image pickup output signal having the same phase with the sampling phase of the red-color picture image pickup output signal and the blue-color picture image pickup output signal or the red-and blue-color picture image pickup output signals having the same phase as the sampling phase of the green-color picture image pickup output signal. In the signal processing circuit of the solid-state imaging device of the present invention, the modulation transfer function (MTF) in the frequency range of from 0 to fs/2 can be increased with the solid-state image pickup sensor having the discrete pixel array and adapted for performing the spatial sampling of the object image as the subnyquist system. Also the aliasing components into the frequency range of from 0 to fs/2 may be cancelled by the processing conforming to the spatial pixel shifting method to produce imaging output signals having superior image quality. Thus, by applying the signal processing circuit to the multi-plate type solid state imaging device for color imaging, it is possible to produce high quality color imaging output signal of high quality containing lesser amounts of folding components.

Claims: A signal processing circuit for a solid-state image pickup device comprising analog-to-digital converting means (6) for digitizing image pickup output signals read out at a sampling rate fs from a solid-state image sensor (4G) for green-color image pickup, from a solid-state image sensor (4R) for red-color image pickup and from a solid-state image sensor (4B) for blue-color image pickup of an image pickup section (1 - 4), at a clock rate equal to said sampling rate fs, said solid-state image sensor (4G) for green-color image pickup being arrayed with a spatial shift of one half of the image sensor pixel pitch with respect to said solid-state image sensors (4R, 4B) for red-color and blue-color image pickup, and interpolating filter (7, 8) means for performing sampling phase conversion for matching the respective sampling phases of the image pickup output signals output by said analog-to-digital converting means (6) at the clock rate fs, characterized in that said interpolating filter means (7, 8) comprises a digital filter operating at the same clock rate as the clock rate of said analog-to-digital converting means (6), said digital filter (7) having a single zero at fs/2 and a positive phase passband in the range from 0 to fs/2, whereby image pickup output signals having a coincident sampling phase are formed at the same clock rate as that of said analog-to-digital converting means (6). The signal processing circuit according to claim 1, wherein said interpolating filter means (7) performs sampling phase conversion of the green-color image pickup output signal digitized by said analog-to-digital converting means (4G). The signal processing circuit according to claim 1, wherein said interpolating filter means (8) performs sampling phase conversion of the red-color and blue-color image pickup output signals digitized by said analog-to-digital converting means (4R, 4B). A color television camera comprising a signal processing circuit according to any one of the preceding claims.

Inventors: ASAIDA TAKASHI, ASAIDA, TAKASHI
Applicants: SONY CORP, SONY CORPORATION
Application year: 1989
Main classification: H04N9
IPC codes: H04N9, H04N5