Stereo camera device and rectification method therefor

A stereo camera device according to one embodiment of the present invention comprises: a light pattern projection unit for projecting a predetermined pattern; a first camera unit for photographing the pattern; a second camera for photographing the pattern; and a control unit for controlling the light pattern projection of the light pattern projection unit, and determining whether to perform rectification between a first image and a second image by analyzing the first image photographed by the first camera unit and the second image photographed by the second camera unit.

TECHNICAL FIELD

The present invention relates to a stereo camera device and a rectification method therefor.

BACKGROUND ART

A stereo camera device is a device that simultaneously receives images from two cameras. The stereo camera device calculates distances from left and right images to an object, and thus may be used to obtain a three-dimensional (3D) image.

Such stereo cameras may be aligned on the same row, and may extract a depth of an object on the assumption that the stereo cameras are aligned on the same row. However, when a tolerance occurs in the production process or positions of the cameras deviate from each other during use, the stereo cameras may not be aligned on the same row. Specifically, it is easy for positions of stereo cameras mounted on a vehicle to be deviated from each other due to an impact and the like.

Thus, in order to obtain an accurate depth extraction result, a rectification process in which left and right images are changed to be like images photographed by two cameras aligned on the same row is required.

A reference point is required for rectification. Generally, the reference point may be extracted from a chess board or a 3D object. However, there are problems in that a space is required to install the chess board or the 3D object and the reference point is difficult to be recognized at night or in places such as a basement and the like, in which an amount of light is insufficient.

Therefore, a method of performing rectification without time and space constraints is required.

Disclosure

Technical Problem

The present invention is directed to providing a stereo camera device and a rectification method therefor.

Technical Solution

One aspect of the present invention provides a stereo camera device. The device includes a light pattern projection unit configured to project a predetermined pattern, a first camera configured to photograph the pattern, a second camera configured to photograph the pattern, and a controller configured to control a light pattern projection of the light pattern projection unit and determine whether to perform rectification between a first image and a second image by analyzing the first image photographed by the first camera and the second image photographed by the second camera.

The controller may drive a timer, and request the light pattern projection from the light pattern projection unit when the timer expires.

The controller may be connected to an electronic control unit (ECU) of a vehicle, and may request the light pattern projection from the light pattern projection unit after receiving an impact sensing signal from the ECU.

The device may further include a rectification unit configured to perform the rectification between the first image and the second image when receiving a rectification request from the controller.

The pattern may be an infrared (IR) pattern.

Another aspect of the present invention provides a rectification method for a stereo camera device. The method includes projecting a light pattern, photographing the projected pattern using a first camera, photographing the projected pattern using a second camera, analyzing a first image photographed by the first camera and a second image photographed by the second camera, and performing rectification when distortion occurs between the first image and the second image.

The light pattern may be projected when a preset timer expires.

The light pattern may be projected when an impact of the stereo camera device is sensed.

Advantageous Effects

According to embodiments of the present invention, rectification can be performed without time and space constraints. That is, there is no need to install or store a three-dimensional (3D) object in order to extract a reference point required for rectification, and rectification can also be performed at night or in a basement, in which an amount of light is insufficient. Further, since whether to perform the rectification is automatically determined before a user senses an abnormality and then the rectification is performed, the user does not need to directly visit a service center and the like, and the quality of an image can be improved.

MODES OF THE INVENTION

While the invention is may have various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will be described herein in detail. It should be understood, however, that there is no intent to limit the invention to the particular forms disclosed, but on the contrary, the invention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting to the invention. As used herein, the singular forms “a,” and “an” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be further understood that the terms “comprise,” “comprising,” “include” and/or “including,” when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

Hereinafter, exemplary embodiments of the invention will be described in detail with reference to the accompanying drawings. Like reference numerals denote like elements or corresponding elements even in different drawings, and thus the description thereof will not be repeated.

FIG. 1is a view illustrating an example in which rectification is performed on a stereo camera device.

Referring toFIG. 1, a reference point is required to perform rectification on two cameras110and120of a stereo camera device100. In order to extract the reference point, a chess board C or a three-dimensional (3D) object should be installed in front of the cameras110and120. In this case, a space for installing the chess board C or the 3D object is required, and a light source L which irradiates the chess board C or the 3D object with light is required.

Thus, there is need to reduce space and time constraints for performing rectification in the stereo camera device.

According to the embodiment of the present invention, the stereo camera device includes a light pattern projection unit, and the light pattern projection unit projects a predetermined light pattern in order to extract the reference point for performing rectification.

FIG. 2is a view illustrating a stereo camera device according to one embodiment of the present invention and a pattern projected therefrom.

Referring toFIG. 2, a stereo camera device200includes two cameras210and220and a light pattern projection unit230. The light pattern projection unit230projects a predetermined pattern P onto a screen or a wall. A reference point for rectification may be extracted from the pattern P projected by the light pattern projection unit230. Accordingly, there is no need to install a chess board or a 3D object, and a separate light source is not required.

Hereinafter, the stereo camera device according to one embodiment of the present invention and a rectification method therefor will be described in detail.

FIG. 3is a block diagram illustrating the stereo camera device according to one embodiment of the present invention, andFIG. 4is a flowchart illustrating a rectification method for the stereo camera device according to one embodiment of the present invention.

Referring toFIG. 3, the stereo camera device200includes a first camera210, a second camera220, the light pattern projection unit230, a controller240, and a rectification unit250.

The first camera210and the second camera220may be aligned to be spaced a predetermined distance from each other on the same row. The first camera210and the second camera220photograph the same image from the left and right. Left and right images respectively photographed by the first camera210and the second camera220may be used to calculate a distance to an object. This method is similar to a principle in which a sense of perspective is felt due to parallax of binocular vision.

Meanwhile, the light pattern projection unit230is located between the first camera210and the second camera220. The light pattern projection unit230may be aligned on the same row as the first camera210and the second camera220. For example, the light pattern projection unit230may be disposed between the first camera210and the second camera220, that is, on a center of a line that connects between the first camera210and the second camera220. When the light pattern projection unit230projects a predetermined light pattern, each of the first camera210and the second camera220photographs the pattern projected by the light pattern projection unit230. In this case, the pattern projected by the light pattern projection unit230may be, for example, a pattern having a chess board shape.

The controller240is connected to the first camera210and the second camera220, and determines whether to perform rectification between a first image and a second image by analyzing the first image photographed by the first camera210and the second image photographed by the second camera220. For example, when it is determined that a row alignment between the first camera210and the second camera220deviates from the first image and the second image, the controller240may transmit a rectification request to the rectification unit250.

When the rectification unit250receives the rectification request from the controller240, the rectification unit250performs rectification between the first image and the second image. That is, the rectification unit250changes the first image and the second image to be like images photographed by the first camera210and second camera220which are aligned on the row. For example, the rectification unit250may make an optical axis and principle points become parallel and cross at infinity. That is, the rectification unit250may perform the rectification so that conditions in which overlap between the first image and the second image is maximized and distortion is minimized are satisfied. The rectification unit250may perform rectification by using, for example, a Hartley algorithm or a Bouguet algorithm. In this case, a method of extracting a pattern of the light pattern projected by the light pattern projection unit230and a reference point in the pattern may vary according to the rectification method of the rectification unit250.

Referring toFIGS. 3 and 4, when the controller240of the stereo camera device200requests light pattern projection from the light pattern projection unit230(S300), the light pattern projection unit230projects a predetermined light pattern (S310). In this case, the light pattern projection unit230may project the predetermined light pattern onto a screen or wall in front thereof. The light pattern may be infrared (IR) light, and a shape of the light pattern may be variously implemented according to a reference point extraction algorithm.

Each of the first camera210and the second camera220photographs the pattern projected by the light pattern projection unit230(S320and S330), and transmits the photographed image to the controller240(S340and S350). In this case, when each of the first camera210and the second camera220senses the projected light pattern, each of the first camera210and the second camera220may automatically photographs the pattern, and may be set to transmit the photographed image to the controller240.

The controller240analyzes the images received from each of the first camera210and the second camera220(S360). To this end, the controller240extracts reference points from each of the images, and determines whether the images are distorted using the reference points.

When it is determined that the images are distorted (S370), the controller240requests rectification from the rectification unit250(S380), and the rectification unit250performs rectification (S390). That is, the rectification unit250may change two images to be like images photographed by cameras aligned on the same row. For example, the rectification unit250may make an optical axis and principle points become parallel and cross at infinity. Accordingly, overlap may be maximized and the distortion may be minimized. In this case, the rectification unit250may perform the rectification using epipolar geometry. In this manner, the rectification unit250may provide high-quality images by correcting errors of a device through software.

Meanwhile, in the stereo camera device according to the embodiment of the present invention, the controller240may automatically control the light pattern projection of the light pattern projection unit230for rectification.

FIG. 5is a flowchart illustrating a method of controlling a light pattern projection of the stereo camera device according to one embodiment of the present invention.

Referring toFIG. 5, the controller240of the stereo camera device starts a timer (S500). The timer may be preset by a user or may be preset according to specifications of the camera.

When the timer expires (S510), the controller240requests a light pattern projection from the light pattern projection unit230(S520).

In this manner, when a preset time has elapsed, the stereo camera device may automatically project a light pattern, and may perform rectification according to whether an image is distorted. Accordingly, the user's convenience may be improved, and the quality of depth extraction may be improved.

Although not illustrated, after the stereo camera device completes the rectification, the stereo camera device may start the timer again.

FIG. 6is a flowchart illustrating a method of controlling a light pattern projection of a stereo camera device according to another embodiment of the present invention. It is assumed that a stereo camera device is mounted on an automobile.

Referring toFIG. 6, an electronic control unit (ECU) of the automobile senses an impact (S600). The ECU is connected to an acceleration sensor, a gyro sensor, an impact sensor, and the like and may sense an impact applied to the vehicle by these sensors.

When the ECU senses the impact, the ECU transmits an impact sensing signal to the controller240of the stereo camera device (S610).

When the controller240receives the impact sensing signal from the ECU, the controller240determines a possibility of positions of two cameras deviating from each other, and requests a light pattern projection from the light pattern projection unit230(S620). However, when the automobile is moving, the light pattern projection unit230may project the light pattern after the automobile stops. To this end, the ECU may transmit the impact sensing signal to the controller240after the automobile stops, or the controller240may request the light pattern projection from the light pattern projection unit230after the automobile stops.

In this manner, when the impact is applied to the vehicle, the light pattern may be automatically projected, and rectification may be performed according to the distortion of an image. Accordingly, since the rectification is automatically performed before the user senses an abnormality, the user's convenience can be improved.

While the example embodiments of the present invention and their advantages have been described above in detail, it will be understood by those of ordinary sill in the art that various changes, substitutions and alterations may be made herein without departing from the scope of the invention as defined by the following claims.