Camera incorporating image shake detection and correction functions

Abstract: A video system according to the present invention defines representative points (3,4) in an image and calculates an absolute value of brightness difference (5) between corresponding points of successive fields so as to extracts moving vector according to the minimum of the difference (6,7). In order to compensate the difference of vertical coordinate of successive fields in the interlaced display, interpolated field lines (10) are calculated in one of successive fields.

Description: SPECIFICATION FIELD OF THE INVENTION The present invention relates to a video system with a camera incorporated therein, particularly to a video system for preventing the shaking of its body. BACKGROUND OF THE INVENTION When a video camera is used on moving body, such as a car, helicopter, etc., the displayed image shakes though the video-camera even when it is fixed to the moving body. As for a integrated system of video and camera ("movie", hereinafter), inputted image often shakes particularly in high zoom magnification when it is used with hands (not fixed). Conventionally, shaking is limited by fixing a camera on a dumping stand for preventing shaking when a movie is used on moving body. In a movie, the visual range is changed in order to prevent shaking of an image by moving a lens in response to the output of an angler velocity sensor. In another type of movie, the shaking is compensated electrically by means of a memory for storing an image. The most typical electrical compensating manner for shaking is: i) individual circuits are provided for odd fields and even fields, respectively, ii)extracting a moving vector on a field, and iii) the following field is compensated to the field on which the moving vector is extracted. Patent Abstracts of Japan, vol. 13, no 371(E-807)[3719], 17.06.89 & JP-A-01 125 064 discloses an image pick-up device compensating for image deterioration due to camera shake. Image signals are picked up by image pick-up elements and converted to digital signals by and A/D-converter. These digital signals are written into a memory and supplied to a "moving length detecting circuit" determining a motion vector using a representative point matching method. On the basis of this motion vector, read-out of data from the image memory is controlled. SUMMARY OF THE INVENTION When a dumping stand for shaking is used, enjoyable or light-hearted usage of movie is impossible because the total size and weight of the instrument is rather big. When an angler velocity sensor is used for a movie, the size of the movie becomes large and the weight of it becomes heavy. Therefore, anyone cannot use it easily: that is, it is not practical. Using the electric compensation for an image of video movie deteriorates the quality of the image. When the information of moving vector is electrically extracted on an image, it is judged whether the image is on odd field or even field, and the moving vector is extracted by using field delay or so between successive odd/even fields. Since the circuits for odd field and even field are to be provided for extracting moving vector independently, a large scale circuit and a lot of electric power is necessary. Therefore, it cannot be applied for a small apparatus such as a movie. The present invention has an object to provide a video system with a camera of simple structure, with small circuit which works by lower electric power. This object is achieved by claims 1 and 2. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 shows a block diagram of moving vector extracting portion. Figs. 2 (a) and (b) show the representative points and searching area for calculating moving vectors. Fig. 3 shows a block diagram of the structure for compensating shaking using field memories. Fig. 4 shows the timing for writing and reading of data in field memories in Fig. 3. Fig. 5 shows an embodiment for controlling the position of reading. Fig. 6 shows a block diagram of a structure for compensation according to the moving vector in CCD. PREFERRED EMBODIMENT OF THE PRESENT INVENTION Hereinafter, an embodiment of the present invention is described with referring to the attached drawings. Fig. 1 shows the fundamental structure for extracting moving vector from sequential field information. In this figure, "1" represents an A/D converter for converting analog signal into digital signal, "2" represents a timing generator for generating a clock for controlling the operation of A/D converter 1 and reading/writing of memories 3 and 4 for representative points, described later, according to an image signal and reference signal. 3 and 4 are the first and second representative point memories for storing a signal of brightness converted from analog to digital by A/D converter 1. The contents of a representative point memory is not transmitted to the other representative point memories. The information delayed by one vertical cycle is stored by switching of a circuit. That is, the signals of brightness delayed by one vertical cycle are stored in the first representative point memory 3 and the second representative point memory 4. "5" represents an absolute difference calculator for calculating the absolute value of the difference between the data converted into digital value and the output data of representative point memory 4. "6" represents an adder for adding the value of brightness in every searching area. "7" is an accumulation memory for accumulating and recording the value of addition in adder 6. "8" and "9" are switches which switch the switch contacts for every vertical cycles. "10" is a field interpolator. The image inputted from CCD (photo device) not shown is converted to digital signal by A/D converter 1. The sampling on this A/D conversion is controlled by the clock generated by timing generator 2. The image signal converted to digital signal is inputted to field interpolator 10. Field interpolator 10 interpolates the image signal during odd field and it outputs the image signal as it is during even field. The "interpolation" here means calculation of mean value between the pixels on the corresponding position of successive 2 field lines. From the calculation, the pixel data on odd field is positioned in alignment with corresponding pixel on even field. The data of the image signal passed through field interpolator 10 is inputted absolute difference calculator 5 and the first representative point memory 3. Representative point memory storages pixel data of search areas, as shown in Fig. 2 (a), each of which includes one representative point and field with density of the representative point. The search area is the area where the moving vector is searched. The representative points are decided in a frame memory beforehand (shown in Fig. 2 (a)). The number of the representative points is calculated according to the number of the pixels in a frame memory and the size of the search area of a moving vector. Here, the number of the representative points and the search area is determined so that the moving vector of each area becomes substantially equal to others. It is assumed here that the data of brightness converted from analog to digital in A/D converter 1 is on odd field, for easy understanding. Then, switch 8 is on "a" terminal and switch 9 is on "b" terminal. Therefore, the data of brightness on odd field is stored on the first representative point memory 3. After one vertical cycle, switch 8 changes to "b" terminal and switch 9 changes to "a" terminal. Then, the data of brightness on even field is stored on the second representative memory 4. The data of brightness on odd field is read out from the first representative point memory 3 and it is inputted to absolute difference calculator 5. Since the brightness value of representative point is known on reading, only the brightness values of the representative points are outputted as the brightness value of points within the search areas. In absolute difference calculator 5, the difference is calculated between the brightness value on even field newly inputted and the output data from representative point memory 3 (brightness value), then, the absolute value is calculated. The calculation of the absolute difference is performed for pixels on the corresponding positions of the even field and output data according to the clock signal sent from timing generator 2. In adder 6, the value after the calculation is added to the data which is read out from accumulation memory 7 and stored again in accumulation memory 7. In the memory 7, the accumulated value of absolute difference is stored between the brightness data in the predetermined field and that in the field inputted after one vertical cycle time. The size of accumulation memory 7 is the same as the number of pixel data in search area (as in Fig. 2 (b)). The vector from the representative point to a point with the lowest brightness in the accumulation memory 7 represent the moving vector. The horizontal and vertical component of the vector is outputted. This method for calculating moving vector is called "representative point matching method". After one vertical cycle time, switches 8 and 9 change to the terminal "a" and "b", respectively. In the same way as above, the brightness on odd field is outputted from A/D converter 1 and that on even field is outputted from the second representative point memory 4. The absolute value of the difference between them is calculated in absolute difference calculator 5, and the value is accumulated and stored in accumulation memory 7. The moving vector is calculated in this way. Fig. 3 shows the structure for compensating the shaking using field memories. In this figure, "31" represents the A/D converter for converting input image signal into digital signal. "32" is a field memory for an odd field, and "33" for an even field. "34" is a D/A converter for converting image signal into analog signal for regenerating an image. "35" and "36" are switches changing terminals for every vertical cycle. "37" shows the moving vector extraction portion which comprises the same circuit as that in Fig. 1. First, switches 35 and 36 are switched on "a" side and "b" side, respectively. The brightness data of an image signal on odd field is converted in A/D converter 31 and the converted data is stored in field memory 32. Field memory 33 is read out at the same time. Next, switches 35 and 36 are switched to the terminal "b" and "a", respectively, and the brightness on even field is inputted to field memory 33. The brightness on odd field stored previously in field memory 32 is read out without compensation, because the moving vector is then being calculated in moving vector extraction portion 37. After that, switches 35 and 36 are switched again to "a" side and "b" side, respectively, and the brightness on odd field is stored in field memory 32. Then, the brightness on even field stored in field memory 33 is read out with compensation, by adjusting the reading out point by the distance according to the moving vector calculated in moving vector extracting portion 37. The compensated brightness is outputted through D/A converter 34, and consequently, the compensated image is recorded and regenerated. Shakingless image can be obtained by repeating such compensation according to the moving vector. Fig. 4 shows the timing chart of writing and reading of data of field memories. In the period 32a, the data on the first odd field is written in field memory 32. On this time, nothing is recorded in 33a. Next in 33b, the data of even field is written in field memory 33. At the same time, in 32b, data stored in field memory 32 before one vertical cycle time is read out, with shifting the read out point according to the movement vector outputted from the vector extraction portion 37. The next data of odd field is recorded on field memory 32 in 32c, and is read out from field memory 33 after compensation in 33c. Shakingless image can be obtained from successive field information by repeating such compensation. Fig. 5 shows a practical example for compensating the location of reading out memory. Without moving vector, reading out of pixel data is started, for example, from the point "P" and scanned in horizontal direction. When moving vector exists, reading out of the pixel data is started, for example, from the point "Q" after the compensation by the value of the moving vector. Fig. 6 shows a structure for compensating shaking of CCD according to the moving vector. In this figure, "61" represents a device CCD for receiving light from lens so as to change the light information into electric signal. "62" is a converting portion for converting the signal from CCD into image signal. "63" is a CCD control portion for controlling clock read out from CCD 61. "64" is a moving vector extracting portion consisting of the circuit shown in Fig. 1. "65" is a compensation vector generating portion, which controls the position from where CCD starts to read out, according to the information from moving vector extracting portion 64. The signal outputted from CCD 61 is converted into an image signal in converting portion 62. It is recorded and regenerated, as well as inputted to moving vector extracting portion 64. Image signal of two fields is inputted to moving vector extracting portion 64, and a moving vector is extracted. In compensation vector generating portion 65, the optimal compensating value is calculated according to the moving vector extracted in moving vector extracting portion 64 and to the moving vector sent from moving vector extracting portion 64 in the previous cycle. The compensating value is inputted to CCD controlling portion 63, and the reading out position of CCD 61 is changed and shaking is compensated. A compensated natural image without shaking can be obtained through it. As mentioned above, moving vector extracting circuits have not to be set independently for off field and even field. Therefore, we can obtain a video system united with camera with simple structure, small circuit and for saving electric power.

Claims: A video system comprising: a camera (61) for scanning an image and outputting corresponding analogue signals; first A/D converter means (31) for converting the analogue signals into digital data of even field and odd field; first and second storing means (32, 33) for storing said digital data of even field and odd field; moving vector extraction means including second A/D converter means (1) for converting the brightness signal part of the analogue signal provided by the camera (61) into brightness digital data, interpolating means (10) for receiving an output from said second A/D converter means means for generating during odd fields, a field consisting of interpolated data, said interpolated data being generated by calculating the average brightness value of corresponding pixels on two successive field lines, a first and a second representative point memory (3, 4) receiving the output of said interpolating means (10), said first representative point memory (3) being for storing brightness digital data of representative points of even fields and said second representative point memory (4) being for storing brightness digital data of representative points of odd fields, or said first representative point memory (3) being for storing brightness digital data of representative points of odd fields and said second representative point memory (4) being for storing brightness digital data of representative points of even fields, difference calculation means (5) for calculating an absolute difference between brightness data of a current field output by the interpolating means (10) and the representative point data of the preceeding field as stored in the first or second representative point memory respectively and means (6, 7, 37) for calculating a moving vector based on the absolute difference supplied by the difference calculating means and for reading out alternately the content of the first and second storing means (32, 33), whereby the read out starting point for even fields is shifted in accordance with the calculated moving vector, and means for recording or reproducing a compensated image based on the content of the compensated even and odd field data. A video system comprising: a camera comprising CCD-means (61) for converting received light into electrical signals, conversion means (62) for converting the signals from the CCD-means into image signals, a moving vector extraction means (64) for generating moving vector information in accordance with the signals provided by the conversion means, a compensation vector generating and controlling portion (65, 63) for controlling, in accordance with the moving vector signal generated by the moving vector extracting portion (64) during the even fields, the starting point, from where the CCD-means read out, wherein said moving vector extraction means includes an A/D converter means (1) for converting the brightness signal of the signal provided by the conversion portion (62) into brightness digital data, interpolating means (10) for receiving an output from said first A/D converter means for generating during odd fields a field consisting of interpolated data, said interpolated data being generated by calculating the average brightness value of corresponding pixels on two successive field lines, a first and a second representative point memory (3, 4), receiving the output of said interpolating means (10), said first representative point memory (3) being for storing brightness digital data of representative points of even fields and said second representative point memory (4) being for storing brightness digital data of representative points of odd fields said first representative point memory (3) being for storing brightness digital data of representative points of odd fields and said second representative point memory (4) being for storing brightness digital data of representative points of even fields, difference calculation means (5) for calculating the absolute difference between brightness data of a current field output by the interpolating means (10) and the representative point data of the preceeding field as stored in the first or second representative point memory respectively and means (6, 7) for calculating a moving vector based on the absolute difference applied by the difference calculating means and for providing said moving vector to said compensation vector generating and controlling portion.

Inventors: MURAKAMI TETSUYA, TAKAHAMA KENGO, MURAKAMI, TETSUYA, TAKAHAMA, KENGO, Murakami, Tetsuya, c/o SHARP CORPORATION, Takahama, Kengo, c/o SHARP CORPORATION
Applicants: SHARP KK, YOZAN INC, SHARP CORPORATION, YOZAN INC.
Application year: 1991
Main classification: H04N5
IPC codes: H04N3, H04N5