Source: http://www.google.com/patents/US5719624?dq=6,907,387
Timestamp: 2016-05-07 01:24:35
Document Index: 685051731

Matched Legal Cases: ['arts 201', 'art 201', 'art 202', 'art 203', 'art 204', 'art 205', 'art 206', 'art 207', 'art 208', 'arts 201', 'art 208', 'arts 202', 'art 204', 'art 201', 'art 201', 'art 205', 'art 206', 'art 207']

Patent US5719624 - Image recording apparatus with arithmetic processing - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inPatentsAn image recording apparatus has a memory for directly storing a color signal supplied from a solid-state image sensor in a memory without applying color signal processing to that color signal, a selecting circuit for selecting block data of arbitrary size from the memory, and a matrix calculation circuit...http://www.google.com/patents/US5719624?utm_source=gb-gplus-sharePatent US5719624 - Image recording apparatus with arithmetic processingAdvanced Patent SearchPublication numberUS5719624 APublication typeGrantApplication numberUS 08/108,840Publication dateFeb 17, 1998Filing dateAug 18, 1993Priority dateJul 18, 1989Fee statusPaidAlso published asUS6515698Publication number08108840, 108840, US 5719624 A, US 5719624A, US-A-5719624, US5719624 A, US5719624AInventorsTakashi Sasaki, Toshihiko MimuraOriginal AssigneeCanon Kabushiki KaishaExport CitationBiBTeX, EndNote, RefManPatent Citations (13), Referenced by (27), Classifications (15), Legal Events (3) External Links: USPTO, USPTO Assignment, EspacenetImage recording apparatus with arithmetic processing
US 5719624 AAbstract
An image recording apparatus has a memory for directly storing a color signal supplied from a solid-state image sensor in a memory without applying color signal processing to that color signal, a selecting circuit for selecting block data of arbitrary size from the memory, and a matrix calculation circuit for converting the selected block data into a predetermined color or luminance signal.
1. An image recording and/or reproducing system comprising:(a) image pickup apparatus including:image pickup means having a color separating optical filter for producing a plurality of color signals; recording means for recording color image signals for one frame and color conversion matrix coefficient information relating to said color separating optical filter together with said color image signals for one frame on a removable recording medium; and (b) reproducing apparatus separate from said image pickup apparatus including:reproducing means for reproducing said color image signals and said color conversion matrix coefficient information from said removable recording medium; and matrix operating means for matrix operating said reproduced color image signals with said reproduced color conversion matrix coefficient information to form predetermined color signals. 2. A system according to claim 1, wherein said recording medium includes a semiconductor memory.
3. A system according to claim 1, further comprising control means for varying said color conversion matrix coefficient information.
4. A system according to claim 3, wherein said control means varies said color conversion matrix coefficient information according to a color filter pattern on an image sensor.
5. An image pickup apparatus comprising:(a) image pickup means having a color separating optical filter for producing a plurality of color signals; and (b) recording means for recording color image signals for one frame and color conversion matrix coefficient information relating to said color separating optical filter together with said color image signals for one frame on a removable recording medium. 6. An apparatus according to claim 5, wherein said recording medium includes a semiconductor memory.
7. An apparatus according to claim 5, further comprising control means for varying said color conversion matrix coefficient information.
8. An apparatus according to claim 7, wherein said control means varies said color conversion matrix coefficient information according to a color filter pattern on an image sensor.
9. A reproducing apparatus comprising:(a) reproducing means for reproducing color image signals for one frame and color conversion matrix coefficient information relating to a color separating optical filter of an image sensor recorded together with said color image signals from a removable recording medium; and (b) matrix operating means for matrix-operating said reproduced color image signals with said reproduced color conversion matrix coefficient information to form predetermined color signals. 10. An apparatus according to claim 9, wherein said recording medium includes a semiconductor memory.
This is a continuation application under 37 CFR 1.62 of prior application Ser. No. 553,896, filed Jul. 13, 1990, now abandoned.
The present invention relates to an image recording apparatus which employs a solid-state image sensor in combination with a color separation filter assembly.
Special attention has recently been directed to an apparatus for converting a signal output from a solid-state image sensor into a digital signal, applying digital signal processing to the digital signal, and recording or reproducing the signal as an image. A typical example of such an apparatus is a full-solid-state camera arranged to convert video information into a digital value and record it on a semiconductor memory or the like. In the aforesaid apparatus, it is necessary to execute various kinds of signal processing, depending on the arrangement of a color separation filter assembly attached to the image sensor (hereinafter referred to as a "sensor"). Accordingly, in an electronic camera, video information is converted into a standard signal form which does not depend on the arrangement of the color separation filters, for example, either a luminance signal and color-difference signals or R, G and B signals, and the thus-converted signals are recorded on a semiconductor memory or the like.
However, the aforesaid conventional arrangement has a problem in that information is recorded on the memory in an amount which is larger than the original amount. For example, in an arrangement in which the output of a sensor having general complementary color mosaic filters is fetched into a memory, the memory capacity required for the luminance signal is needed by an amount approximately corresponding to the number of pixels on the sensor since a sampling frequency cannot be reduced. In addition, since it is necessary to convert color-difference signals which are originally obtained in a line-sequential manner into simultaneous signals to be stored, a memory capacity needs to be several times as large as the original amount of information supplied from the sensor. Of course, it is possible to lower the sampling frequency of each of the color-difference signals. However, in order to lower such sampling frequency, a digital filter is needed. Furthermore, since the sampling frequency cannot be lowered below a certain limit, it is difficult to prevent the required memory capacity from increasing to a great extent above the original amount of information supplied from the sensor. The above problem is also experienced with a sensor having pure color stripe filters which will be described later. In order to store the original information on the memory without increasing the amount of information, it is necessary to directly record the information supplied from the sensor. However, since different kinds of color filters require different forms of signal processing, the problem that interchangeability is not assured remains to be solved.
It is, therefore, an object of the present invention to provide an image recording apparatus which is capable of solving the above-described prior art problems.
To this end, according to one embodiment of the present invention, there is provided an image recording apparatus which comprises a memory for directly storing a color signal supplied from a solid-state image sensor in a memory without applying color signal processing to that color signal, selecting means for selecting block data of arbitrary size from the memory, and a matrix calculation circuit for converting the selected block data into a predetermined color or luminance signal.
In a specific form of the aforesaid embodiment, a reproducing apparatus is provided with an n�m matrix signal processing means for converting a signal supplied from a sensor into R, G and B signals as well as a luminance signal. In recording mode, a recording apparatus is adapted to directly store the signal output form the sensor in its memory and to record on the memory the requisite information such as a conversion matrix constant, information on the lattice arrangement of the sensor, and the number of steps of horizontal and vertical movements. Accordingly, it is possible to produce a signal output having a common form with respect to an arbitrary arrangement of colors.
In accordance with the above-described embodiment, if the signal output from the sensor is substantially directly written to the memory, interchangeability is not impaired and outstanding memory saving can be achieved.
In accordance with another embodiment of the present invention, there is provided an electronic camera which is arranged to record on a detachable memory an image signal and information indicative of the characteristics of an image sensor.
In this embodiment, it is possible to record the output of the image sensor in substantially unprocessed form, whereby image information can be recorded with the amount of information per image reduced. In addition, since information on the property of the image sensor is stored, the stored image information can be readily converted into image information having interchangeability in a final step.
FIG. 1 is a view showing one example of the arrangement of complementary mosaic filters and corresponding matrix expressions;
FIG. 3 is block diagram showing one example of an arrangement according to the present invention;
FIG. 4 is a view showing one example of the arrangement of pure color stripe filters and corresponding matrix expressions;
FIG. 5 is a view showing another example of the arrangement of pure color stripe filters and a corresponding luminance signal matrix expression;
FIG. 6 is a block diagram showing the arrangement of one embodiment of an image recording apparatus according to the present invention;
FIG. 7 is a view showing the configuration of a memory space in the embodiment of the present invention;
FIG. 8 is a view showing another example of a window matrix according to the embodiment of the present invention;
FIG. 9 is a block diagram showing one example of a color signal processing matrix circuit for use in the embodiment of the present invention;
FIG. 10 is a view showing one example of a color signal matrix expression which corresponds to the circuit of FIG. 9;
FIG. 11 is a view showing one example of a luminance signal processing matrix circuit for use in the embodiment of the present invention;
FIG. 12 is a block diagram showing the arrangement of another embodiment of an image recording apparatus according to the present invention;
FIGS. 13 (A), 13 (B) and 13 (C) are views showing different examples of the arrangement of color filters;
FIG. 14 is a view showing the contents of the memory used in the embodiment of FIG. 12;
FIGS. 15 (A) and 15 (B) are views showing the contents of a part of the memory shown in FIG. 14;
FIG. 16 is a block diagram showing a reproducing apparatus according to another embodiment of the present invention; and
FIG. 17 is a flowchart showing the routine executed by the reproducing apparatus of FIG. 16.
First of all, the principle of matrix signal processing which constitutes the gist of each preferred embodiment will be explained below. In general, electronic cameras, whether single-CCD or two-CCD types, utilize limited kinds of color separation filters: complementary color filters and pure color filters. For this reason, such an electronic camera is arranged to conduct signal processing which includes the steps of separating the output from an image sensor into R, G and B signals, white balancing each of the R, G and B signals and converting them into the required signal form.
For example, in an arrangement using the complementary color mosaic filters shown in FIG. 1, a signal processing circuit such as that shown in FIG. 2 is used to effect the following conversions by means of a color signal processing system and a luminance signal processing system, respectively. In the color signal processing system, the output of the image sensor 1 is converted into signals r, g and b by a matrix circuit A. The signals r, g and b are passed through a matrix circuit B having the function of adjusting white balance, a gamma correcting circuit 2 and a color-difference matrix circuit 3, and are in turn supplied to a standard television signal processing encoder 4 which corresponds to NTSC standards, PAL standards, SECAM standards, high definition television (HD) standards or the like. In the luminance signal processing system, the output of the image sensor 1 is converted into a luminance signal by a matrix circuit D and a high-frequency component is extracted from the luminance signal through a gamma correcting circuit 5 and a digital filter (HPF) 6. The high-frequency component and a low-frequency luminance signal supplied from the matrix circuit 3 are synthesized as a luminance signal Y by an adder 7. As is apparent from the foregoing, in order to obtain the required color signal from the image sensor 1, it suffices to utilize matrix information on, a G=A�B color signal matrix, whereas, to obtain the corresponding luminance signal, it suffices to provide the matrix circuit D. In actual signal processing, it is further necessary to provide vertical and horizontal subsampling steps. When expressed in another way, since matrix calculations are performed while an n�m matrix is being superimposed on each pixel of the image sensor 1, information on the next pixel position to which the n�m matrix is to be moved is needed. The movement of such matrix which is associated with the matrix calculations will now be described with reference to FIG. 1. It is assumed here that the signals from the respective pixels of the image sensor 1 are arrayed in an x�y matrix on a memory (RAM) so that the signals can be randomly accessed or so that arbitrary signals can be extracted as the data required for matrix calculations by means of a delay line or the like. To begin with, the window matrix Gc shown in FIG. 1 is set to the coordinate points (1, 1) on the x�y matrix, and the matrix calculations shown in FIG. 1 are used to perform calculations on the signals r, g, b and Y. Then, the window matrix Gc is moved in accordance with a sampling step which is prepared in advance. For instance, in the example shown in FIG. 1, the window matrix Gc, which is first located at the coordinate point (1, 1), is moved to the coordinate point (3, 1) after two cycles. This movement is accomplished in accordance with a reset subsampling step =2. If the area of the x�y matrix is preset, the movement of the window matrix Gc can be stopped at the instant when a coordinate point outside the x�y matrix is reached. After a cycle corresponding to one horizontal scanning period has been passed, the x coordinate is reset to its initial position and the window matrix Gc is moved along the y coordinate axis in accordance with a vertical sampling step, whereby signal processing for the next horizontal line is performed.
The processing of luminance signals is performed in similar manner. Matrix calculations are performed by using the signal processing matrix circuit D which is prepared in association with the window matrix Gc, thereby providing a luminance signal. The window matrix Gc is likewise moved in accordance with the subsampling step so that signal processing similar to that performed on the aforesaid color signal is achieved. As described above, since the window matrix Gc and the signal processing matrix D are prepared in circuit with each other, signals supplied from a mosaic filter assembly and a stripe filter assembly, that is, pure colors and complementary colors, which have conventionally been processed according to different procedures, can be processed merely by switching matrices on an identical hardware arrangement. FIG. 3 is a block diagram showing a signal processing arrangement including window matrices according to the present invention. In FIG. 3, the same reference numerals are used to denote the like or corresponding elements shown in FIG. 2, and the illustrated random access memory (RAM) may be of a fixed or detachable type. FIG. 4 shows one example of a pure color stripe filter assembly and the corresponding matrix. The filter assembly and matrix shown in FIG. 4 will be described below with reference to the hardware arrangement shown in FIG. 3. In the case of the pure color filter, as shown in FIG. 4, the matrix A of FIG. 4 assumes a 3�3 unit matrix, while the luminance signal Y is represented as a 3�1 matrix. This arrangement may also be utilized when the luminance signal Y is to be produced by adding a line signal in order to increase the sensitivity of a sensor incorporated in an electronic movie camera or the like. In this arrangement, as shown in FIG. 5, it is only necessary that the window matrix Gc be expanded in the y direction and that the elements arrayed in the matrix D be increased correspondingly. Accordingly, a signal y1 or y2 may be selected for each field.
In the aforesaid arrangement, the initial coordinates of the window matrix for each field may of course be prepared as coordinate information so as to reduce the matrix. In the case of the luminance signal system, if the size of the window matrix is increased, a vertical enhancer signal can also be output. As a matter of course, it is possible to provide the luminance matrix with the above-described operation.
FIG. 6 is a block diagram of one embodiment of the present invention, and shows one example of a full-solid-state camera which utilizes the aforesaid principle of the present invention, and the operation and arrangement of this camera will be described below. The illustrated camera comprises an optical system 51, a stop 52, an infrared cut filter 53, an optical low-pass filter 54, and a one-chip color separation filter 55. An image, passed through the elements 51 to 55, is formed on a solid-state image sensor 56 using a CCD or the like. The image is converted into electrical charge by the solid-stage image sensor 56. The electrical charge thus obtained is read from the same in synchronism with a signal supplied from a system signal generator (SSG) 77, and is supplied to a variable gain amplifier 57. The variable gain amplifier 57 serves to correct the sensitivity of the solid-stage image sensor 56 which may not be completely adjusted by the stop 52. Simultaneously, the variable gain amplifier 57 effects knee correction. An optical black clamping circuit 58 fixes the black level of the solid-stage image sensor 56. The signal whose black level is fixed as a direct current by the optical black clamping circuit 58, is converted into a digital value by an A/D converter 59. The digital values thus obtained are sequentially stored in a memory 60 in accordance with an address indicated by a controller 78. The memory 60 may be disposed in either a fixed form or a detachable form. An average value detecting circuit 76 is arranged to provide exposure control by utilizing the image sensor output. The average value detecting circuit 76 reads an average value to control the stop 52 and the variable gain amplifier 57, thereby adjusting an exposure level. In the above-described embodiment, an electronic shutter operation is achieved by controlling the timing of driving of the image sensor 56. Also, the above-described embodiment employs a recording format such as that shown in FIG. 7 for writing to the memory 60. As shown in FIG. 7, a header file stores in numerical form the number of recorded images, the titles of the respective images, the property of information other than image information. An ID file part is defined at the head of each image signal recording area. Recorded on the ID file part are various kinds of information such as matrix information supplied from signal processing matrix circuits 61 and 62, the lattice configuration of the solid-state image sensor 56, and the information required for the window matrix, such as subsampling steps. In an area which follows the ID file part, corresponding image data are recorded.
For reproduction, a system controller 79 reads the contents of the ID file part from the memory 60 by means of the controller 78, and obtains matrix constants and a digital filter constant. The system controller 79 transfers the matrix constants to the respective signal processing matrix circuits 61 and 62, and the digital filter constant to a digital filter 75. The system controller 79 also transfers window matrix information or subsampling-step information to the memory controller 78. Thus, the required reproduction conditions are established.
During reproduction, the memory controller 78 generates a window matrix and transfers information corresponding to the position of the window matrix to the color signal processing matrix circuit 61 and the luminance signal processing matrix circuit 62 in accordance with subsampling step. The matrix circuits 61 and 62 perform matrix signal processing on the basis of the matrix information transferred from the ID file part, thereby forming color-difference signals and a luminance signal, respectively. The luminance signal thus formed is then supplied to the digital filter 75, where a high-frequency component yH is extracted. In the meantime, a low-frequency luminance signal yL is obtained by subjecting the output of the memory 60 to calculations in the color signal processing matrix circuit 61 and passing the result through a correcting circuit 84 and a matrix circuit 85. The high-frequency component yH and the low-frequency luminance signal yL are supplied to an adder 74, where they are synthesized with each other. The output of the adder 74 is subjected to blanking processing in a blanking circuit 65 in accordance with a signal which is supplied from a system signal generator 80 during reproduction. The output of the blanking circuit 65 is supplied to an adder 73, where a synchronizing signal is added to the input signal. The output of the adder 73 is converted into an analog signal by a D/A converter 68, and a subsampling carrier is cut by a low-pass filter 70. The output of the low-pass filter 70 is provided as a luminance signal by an amplifier 72.
Also, the color-difference signals output from the matrix circuit 85 are combined with blanking pulses in blanking circuits 63 and 64, respectively. The outputs of the respective blanking circuits 63 and 64 are converted into a chrominance signal in an NTSC encoder 66. The chrominance signal is converted into an analog signal in a D/A converter 67, and the analog signal is output as a color signal C through a band-pass filter 69 and an amplifier 71. In the above-described arrangement, a digital filter may be disposed after the color signal processing matrix circuit 61.
FIG. 8 shows a 3�4 matrix which constitutes a part of the n�m matrix according to the present invention. The required data, which is obtained from a random access memory, is made to correspond to the elements d1 to d12 of the signal processing matrix in the window matrix. The resultant signal form is a one-bit signal form such as that shown in FIG. 8. Relevant data is selected by using the one-bit signal form, and information for setting start coordinates and information on sampling steps are also needed as signals for controlling this window matrix.
The thus-obtained signal is supplied to a 3�12 signal processing circuit such as that shown in FIG. 9, subjected to matrix calculations such as those shown in FIG. 10, and output as the signals r, g and b. The values of constants a1 to a36 take on values which are inherent in the image sensor and which allow for white balance information, a signal processing method, the characteristics of the color separation filter assembly and the like.
Although in the above description the window matrix is an n�m matrix, the window matrix is not limited to such a rectangular configuration. For example, a window having an arbitrary configuration consisting of λ elements may be employed. This explanation also applies to a matrix for luminance signal processing.
FIG. 11 shows an example of a luminance signal processing matrix circuit, and the illustrated example is a 2�12 matrix. The signals y1 and y2 represent the results obtained by effecting signal processing for first and second fields, respectively. These signals y1 and y2 are switched by a multiplexer MPX for each field and are alternately output. The corresponding constants b1 to b24 are constants which indicate the white balance of the luminance signal system, luminance difference correcting information, and information on the vertical enhancer signal or the like.
The luminance signal matrix can also be constructed as a k�j matrix, where k and j are natural numbers, respectively, and the present invention is not limited to the 2�12 matrix used in the above-described embodiment. If the k�j matrix and the m�n matrix are identical to each other, a common latch may be disposed before the corresponding signal processing circuits. In addition, if the signal processing circuits can operate at sufficiently high speeds, a common signal processing circuit which can operate on a time-shared basis may be employed.
Although the above-descrived embodiment is arranged so that the required data are directly stored in the memory RAM, the data may of course be stored in a compressed form and reproduced in an expanded form.
The above-described embodiment has a number of advantages. For example, signal processing circuits, which have conventionally depended on the arrangements of individual color separating filters for use with an image sensor, can be constructed as a common circuit. Accordingly, if such a signal processing circuit is formed as an IC, mass-producibility can be improved. In addition, in a solid-stage camera which is produced in accordance with the present invention, even if a sensor signal is directly stored in a memory, interchangeability is not impaired. Accordingly, the sensor signal can be directly recorded in the memory, whereby the utilization efficiency of the memory can be improved.
FIG. 12 is a block diagram showing the arrangement of an electronic camera to which another embodiment of the present invention is applied. In this embodiment, the information required to perform the processing, such as matrix calculations for reproduction, executed in the above-described embodiment is stored in a memory.
A CCD 101 is equipped with color separation filters such as those shown in FIGS. 13 (A) to 13 (C). A driver circuit 105 reads image signals for each frame from the CCD 101 in accordance with a timing signal supplied from a system signal generator (SSG) 106. The signal output form the CCD 101 is subjected to analog processing, such as correlated double sampling (CDS) and automatic gain control (AGC), in an analog processing section 102. The processed signal is converted into a digital signal in an A/D converter 103.
The digitized signal is compressed by a DPCM or DTC technique in a compressing section 104.
A single image is recorded on a detachable semiconductor memory 111 in the following manner.
First of all, a system controller 109 switches the switch 110 so that the information on the CCD 101 which is output from the system controller 109 can be written to the semiconductor memory 111.
The information is, as shown in FIG. 14, written to header parts 201 to 208 which correspond to a single image.
Written to the header part 201 is the frequency of a horizontal clock pulse for driving the CCD 101.
Written to the header part 202 is the number of pixels of the CCD 101 which are arranged in each of the horizontal and vertical directions.
A method of driving the CCD 101 is written to the header part 203. The contents to be written include encoded information on whether an interlace method or a non-interlace method has been selected, whether frame recording or field recording has been selected, whether frame recording or field recording has been selected, and so on.
Written to the header part 204 are the start position of a light shielding portion (optical black) provided on the CCD 101 and the end position of the same.
Information representing the kind of color filter assembly is written to the header part 205 in the manner shown in FIG. 15(A).
A number CN of the kinds of color filters is written to a portion 301, and the number UX of pixels arranged in the horizontal direction is written to a portion 302 in the repetition unit U of each color filter, while the number Uy of pixels arranged in the vertical direction is written to a portion 303 in the same unit. A portion 304 includes UX �Uy elements, and data on which of color filters 1 to CN corresponds to the U(i, j) is written by scanning form top left corner to top right corner of the repetition unit U.
FIG. 15(B) shows the specific contents of information to be written in the arrangement shown in FIG. 15(A) when the color arrangements of FIGS. 13(A) to 13(C) are employed. Each symbol "-" represents that nothing is to be written. Although the names and numbers 1 to CN of individual colors may be arbitrarily determined, one-to-one correspondence is needed between these numbers and the order of writing coefficients for reproducing primary colors.
Matrix coefficients suitable for use in converting color signals corresponding to the CN color filters to R, G and B signals by matrix calculations are written to the header part 206. For example, in the filter arrangement shown in FIG. 13(A), it is assumed that the following relation is obtained: ##EQU1## In FIG. 15(B), Mg, Gr, Cy and Ye correspond to 1, 2, 3 and 4, respectively. Accordingly, twelve data are written as the color matrix coefficients in the order:
a11 &#8594;a12 &#8594;a13 &#8594;a14 &#8594;a21 &#8594;a22 &#8594;a23 &#8594;. . . &#8594;a34.
A correlative color temperature, measured when an image of interest is recorded, is estimated from color temperature information obtained by converting the output of an external light sensor 107 from analog form to digital form. The correlative color temperature is written to the header part 207.
Written to the header part 208 is a code which represents the kind of algorithm used for compression.
When the above-described operations have been completed, the system controller 109 switches the switch 110 to write the compressed image data to the detachable semiconductor memory 111 by driving the CCD 101 at an appropriate timing.
FIG. 16 is a block diagram showing the arrangement of an apparatus for processing the thus-written data and reproducing it as an image.
The detachable semiconductor memory 111 is connected to a central processing unit (CPU) 501 through an appropriate interface 504.
The CPU 501 performs a reproducing operation in accordance with the flow chart of FIG. 17 which is stored as software.
In Step 601, the CPU 501 reads the various kinds of information on the characteristics of the CCD 101 from the header parts 201 to 208 shown in FIG. 14.
In Step 602, the CPU 501 performs decoding (expansion) which corresponds to the compression algorithm written to the header part 208, and decodes the compressed image data into original image data P(X, Y). In this step, the numbers of pixels in the respective horizontal and vertical directions and codes indicating the kind of driving which are written to the header parts 202 and 203, are indispensable.
Then, in Step 603, the CPU 501 performs software clamping. If the leading and trailing addresses of the optical black written to the header part 204 are represented by Xs and Xe, respectively, the following calculations are performed: ##EQU2## Thus, the software clamping is performed.
Then, in Step 604, the CPU 501 generates a luminance signal L (X, Y) from the array shown by Equation (2).
Calculations for use in producing the luminance signal L(X,Y) from the original image data P(X, Y) include low-pass filtering H1 (X, Y) and high-frequency emphasis filtering H2 (X, Y) used for contour correction.
L(X,Y)=H1 (X, Y)xp(X, Y)+&#945;H2 (X, Y)xP(X, Y)(3)
"x" represents convolution calculations, and the following equation is obtained: ##EQU3## where i and j are varied within the range of H(X, Y).
For example, the low-pass filtering H1 (X, Y) may be one-dimensional filtering of the form:
H1 (X, Y)=1/25 1, 2, 3, 4, 5, 4, 3, 2, 1!             (5)
In this case, since H(X, Y)=0 and Y=1, it suffices to vary i from 1 to 9 for j=1.
It is desirable that the high-frequency emphasis filtering H2 (X, Y) be two-dimentional filtering of the form: ##EQU4##
The frequency characteristics obtained when the respective calculations are carried out vary with a clock frequency fC which is written to the header part 201.
For instance, for the clock frequency fC, the frequency characteristic H(f) of the low-pass filtering H1 (X, Y) expressed by Equation (5) is given by
H1 (f)=1/25(5+8 cos &#969;+6 cos 2&#969;+4 cos 3&#969;+2 cos 4&#969;)                                                 (7)
where ω=2πf/fC.
As can be seen from Equation (7), H1 (f) varies with fC and, therefore, as fC varies, the coefficient of H1 (X, Y) needs to be varied so that substantially the same frequency characteristics can be steadily obtained. Accordingly, in Step 604, information on the clock frequency written to the header part 201 is indispensable. If a user actuates a keyboard 502, the strength α of the high-frequency emphasis in Equation (3) can be arbitrarily adjusted from the outside.
The keyboard 502 may be replaced with other external adjustment means such as a volume.
Finally, the following gamma conversion may be performed: ##EQU5## Where, if L(X,Y) consists of N bits, MAX is 2N -1.
In Step 605, primary-color separating signals RL (X, Y), GL (X, Y) and BL (X, Y) are calculated from the original image data P(X, Y).
For example, in the arrangement shown in FIG. 13(A), Mg (X, Y) is calculated.
The information on the arrangement of unit colors, which is written to the header part 205, is used to set to zero all the data other than the data corresponding to the color filter No. 1 in P(X, Y), thereby producing Mg '(X, Y).
Then, Mg (X, Y) is obtained by the convolution of Mg '(X, Y) and interpolation filtering F(X, Y).
Mg (X, Y)=F(X, Y)xMg '(X, Y)                     (9)
For example, in the arrangement shown in FIG. 13(A), it is preferable that the interpolation filtering F(X, Y) be of the form: ##EQU6##
Similarly, Gr (X, Y), Cy (X, Y) and Ye (X, Y) are each obtained.
Then, matrix coefficient information (ai, j) written to the header part 206 is used to perform the following calculations: ##EQU7##
In Step 606, the CPU 501 executes white-balance control, that is to say, the CPU 501 multiplies RL (X, Y), GL (X, Y) and BL (X, Y) by the relative-color-temperature information read from the header part 207, as follows: ##EQU8## where T represents the relative color temperature and, as T rises, GR(T) increases but GB(T) decreases. GG(T) may be made one.
In Step 607, the CPU 501 calculates color-differences C1 (X, Y) and C2 (X, Y).
The CPU 501 performs calculations represented by
YL (X, Y)=0.30RL (X, Y)+0.59GL (X, Y)+0.11BL (X, Y)(13)
C1 (X, Y)=YL (X, Y)-RL (X, Y) C2 (X, Y)=YL (X, Y)-BL (X, Y)                                         (14)
In Step 608, the CPU 501 uses the calculation results obtained in Steps 604 and 607 to calculate
R(X, Y)=-C1 (X, Y)+L(X, Y) B(X, Y)=-C2 (X, Y)+L(X, Y) G(X, Y)=1/0.59 L(X, Y)-0.30R(X, Y) -0.11B(X, Y)!               (15)
In Step 609, the CPU 501 writes the calculation results R(X, Y), G(X, Y) and B(X, Y) to a frame memory 508 shown in FIG. 16.
These data R(X, Y), G(X, Y) and B(X, Y) are converted by D/A converters 505, 506 and 507, respectively, thereby providing corresponding analog R, G and B signals.
In accordance with the embodiment described above, it is possible to assure the interchangeability of different kinds of image sensors among systems in which data is directly written. Accordingly, it is possible to realize an electronic camera provided with a low-cost, small-size and detachable memory which can record an increased number of images.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS4643563 *Jul 25, 1984Feb 17, 1987Canon Kabushiki KaishaColor image data processing methodUS4755879 *Jun 15, 1987Jul 5, 1988Quantel LimitedVideo processing systemsUS4817175 *May 4, 1987Mar 28, 1989Schlumberger Systems And Services, Inc.Video stream processing systemUS4855814 *Jun 24, 1988Aug 8, 1989Canon Kabushiki KaishaVideo camera body and detachable lens each containing a memory for storing signals indicative of spectral characteristicsUS5018219 *Feb 6, 1989May 21, 1991Hitachi, Ltd.Object recognize apparatusUS5032927 *Mar 1, 1989Jul 16, 1991Fuji Photo Film Co., Ltd.Image signal recording apparatus capable of recording compressed image data together with audio dataUS5034804 *Dec 23, 1988Jul 23, 1991Kabushiki Kaisha ToshibaElectronic still camera with various modes of data compressionUS5142356 *Nov 7, 1991Aug 25, 1992Canon Kabushiki KaishaColor image reading apparatus or color image forming apparatus capable of performing color adjustmentUS5202756 *Jul 16, 1992Apr 13, 1993Canon Kabushiki KaishaColor signal processing apparatus using plural luminance signalsUS5237401 *May 2, 1991Aug 17, 1993Mitsubishi Denki Kabushiki KaishaColor image reading apparatus including color correction meansUS5313274 *Aug 20, 1991May 17, 1994Fujitsu LimitedColor signal matrix circuitUS5428389 *Aug 12, 1993Jun 27, 1995Fuji Photo Film Co., Ltd.Image data storage/processing apparatusUS5493409 *Oct 14, 1994Feb 20, 1996Minolta Camera Kabushiki KaishaStill video camera having a printer capable of printing a photographed image in a plurality of printing modes* Cited by examinerReferenced byCiting PatentFiling datePublication dateApplicantTitleUS6034724 *Feb 3, 1997Mar 7, 2000Chinon Kabushiki KaishaImaging device utilizing a line-crawling correction coefficientUS6515698 *Aug 4, 1997Feb 4, 2003Canon Kabushiki KaishaImage recording apparatusUS6614473 *Oct 2, 1998Sep 2, 2003Olympus Optical Co., Ltd.Image sensor having a margin area, located between effective pixel and optical black areas, which does not contribute to final imageUS6686960 *Feb 17, 1999Feb 3, 2004Sony CorporationMethod for driving an imaging device and image pickup device wherein signals corresponding to same color outputs are combinedUS7212686 *Mar 7, 2003May 1, 2007Sony CorporationImage processing apparatusUS7256831Dec 15, 2003Aug 14, 2007Sony CorporationMethod for driving an imaging device and image pickup device, wherein signals corresponding to same color outputs are combinedUS7307660Dec 18, 2006Dec 11, 2007Sony CorporationMethod for driving solid-state image pickup apparatus, solid-state image pickup device and cameraUS7356204Feb 4, 2005Apr 8, 2008Canon Kabushiki KaishaImage processing apparatus and method of controlling same, computer program and computer-readable storage mediumUS7424171Feb 3, 2005Sep 9, 2008Canon Kabushiki KaishaImage processing method and apparatus, computer program, and computer readable storage mediumUS7466351Mar 18, 2004Dec 16, 2008Canon Kabushiki KaishaInformation processing method, apparatus, program and storage medium storing said program for digital image data generated from an image sensing deviceUS7620271Nov 17, 2009Canon Kabushiki KaishaImage processing apparatus and method of controlling same, computer program and computer-readable storage mediumUS8045022 *Jun 12, 2008Oct 25, 2011Fujifilm CorporationImaging apparatus for correcting a luminance signalUS8068686 *Nov 29, 2011Olympus CorporationImage recording apparatus, image recording method, image processing apparatus, image processing method and image processing systemUS8094966May 14, 2008Jan 10, 2012Canon Kabushiki KaishaImage processing method and apparatus, computer program, and computer-readable storage mediumUS8265411 *Sep 11, 2012Olympus CorporationImage recording apparatus, image recording method, image processing apparatus, image processing method and image processing systemUS8483509Dec 13, 2011Jul 9, 2013Canon Kabushiki KaishaImage processing method and apparatus, computer program, and computer-readable storage mediumUS20030210829 *Mar 7, 2003Nov 13, 2003Ikuo SomeyaImage processing apparatusUS20040196381 *Mar 23, 2004Oct 7, 2004Canon Kabushiki KaishaImage processing method and apparatusUS20040227824 *Mar 18, 2004Nov 18, 2004Kenji TakahashiInformation processing method, apparatus, program and storage medium storing said programUS20050174449 *Feb 3, 2005Aug 11, 2005Canon Kabushiki KaishaImage processing method and apparatus, computer program, and computer-readable storage mediumUS20050175260 *Feb 4, 2005Aug 11, 2005Canon Kabushiki KaishaImage processing apparatus and method of controlling same, computer program and computer-readable storage mediumUS20050280716 *May 31, 2005Dec 22, 2005Tetsuya ToyodaImage recording apparatus, image recording method, image processing apparatus, image processing method and image processing systemUS20070097244 *Dec 18, 2006May 3, 2007Sony CorporationMethod for driving solid-state image pickup apparatus, solid-state image pickup device and cameraUS20080218605 *May 14, 2008Sep 11, 2008Canon Kabushiki KaishaImage processing method and apparatus, computer program, and computer-readable storage mediumUS20090009637 *Jun 12, 2008Jan 8, 2009Tetsu WadaImaging apparatusUS20110304743 *Dec 15, 2011Tetsuya ToyodaImage recording apparatus, image recording method, image processing apparatus, image processing method and image processing systemEP0994629A1 *Sep 17, 1999Apr 19, 2000O.M.S. - S.p.A. Officine Meccaniche SpecializzateMethod for high-quality image acquisition and device for implementing the method* Cited by examinerClassifications U.S. Classification348/231.3, 348/273, 348/E09.01, 386/E05.072International ClassificationH04N5/907, H04N5/77, H04N9/04, H04N9/804Cooperative ClassificationH04N9/8042, H04N5/772, H04N2209/046, H04N9/045, H04N5/907European ClassificationH04N9/04B, H04N5/77BLegal EventsDateCodeEventDescriptionJul 26, 2001FPAYFee paymentYear of fee payment: 4Jul 20, 2005FPAYFee paymentYear of fee payment: 8Jul 15, 2009FPAYFee paymentYear of fee payment: 12RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services