Patent Description:
Development and improvement of automatic driving, automatic braking, automatic following, and the like in automobiles have been rapidly advanced. In order to realize fully automatic driving, it is necessary to accurately recognize an object moving at a high speed and measure a distance to the object in a wide range around the vehicle.

For example, in a case where an object (for example, a motorcycle) moving at <NUM>/h is recognized by a sensor at a front end of the vehicle, it is assumed that imaging with an angular resolution of <NUM>°/pixel and a distance measurement system of <NUM>% at a distance of <NUM> are necessary. Regarding such a requirement, existing laser imaging detection and ranging (LiDAR) and millimeter wave radars have a problem in that it is necessary to use a camera together for accurate recognition of the surroundings. In addition, in a case of a scanning type detection device, since a movable portion is required, there is a problem in that it is expensive and the response speed is not sufficient.

In addition, a stereo camera capable of three-dimensionally grasping the position of an object is also used for automatic driving control and the like. However, in a case of a stereo camera in the related art, it is usually necessary to mount two image sensors apart from each other by an interval corresponding to parallax, which causes an increase in cost and an increase in size. Further, since the size is large, it is difficult to arrange it at the left and right front ends of the automobile, and also difficult to perform wide-angle sensing.

As described above, in a case where a stereo camera is mounted on an automated driving vehicle, a compact stereo camera capable of performing high-accuracy and wide-angle sensing has been required.

<CIT> describes an apparatus for providing panoramic stereo image with single camera. <CIT> refers to a single camera based omnibearing stereo vision system. <CIT> discloses a wide-angle imaging device suitable for a monitor area of about <NUM> degrees. <CIT> refers to an image processing apparatus that detects an object from images taken by an imaging element mounted on a moving body.

An object of the present invention is to provide a compact stereo camera and a light unit with integrated stereo camera capable of performing high-accuracy and wide-angle sensing.

In order to solve the above problem, the subject matter of the independent claims is provided. The dependent claims describe optional embodiments of the invention.

According to the present invention, it is possible to provide a compact stereo camera and a headlight unit with integrated stereo camera capable of performing high-accuracy and wide-angle sensing.

Hereinafter, the present embodiment will be described with reference to the accompanying drawings. In the accompanying drawings, functionally same elements may be denoted by the same numbers. Note that, although the accompanying drawings illustrate embodiments and implementation examples conforming to the principles of the present disclosure, these are for understanding the present disclosure and are not used to interpret the present disclosure in a limited manner. The description herein is exemplary only and is not intended to limit the claims or applications of the disclosure in any way.

In the present embodiment, the description has been made in sufficient detail for those skilled in the art to implement the present disclosure; however, it is necessary to understand that other implementations and embodiments are possible, and changes in configurations and structures and replacement of various elements are possible.

With reference to a schematic perspective view of <FIG>, a stereo camera <NUM> according to a first embodiment will be described with reference to <FIG>. As an example, the stereo camera <NUM> includes an upper hyperboloid mirror <NUM>, a lower hyperboloid mirror <NUM>, an imaging optical system <NUM>, an image sensor <NUM>, a display <NUM>, and a drive control unit <NUM>. One imaging optical system <NUM> and one image sensor <NUM> may be provided for each of a pair of the hyperboloid mirrors (upper hyperboloid mirror <NUM> and lower hyperboloid mirror <NUM>). Since the information on the position of the subject in a three-dimensional space can be grasped by the single image sensor <NUM>, the cost can be reduced as compared with the stereo camera in the related art.

The upper hyperboloid mirror <NUM> (first mirror) and the lower hyperboloid mirror <NUM> (second mirror) integrally constitute a hyperboloid mirror. The upper hyperboloid mirror <NUM> has a reflective surface (first reflective surface) along an upper hyperboloid convex in a downward direction (negative direction of the Z-axis (first direction)), while the lower hyperboloid mirror has a reflective surface (second reflective surface) along a lower hyperboloid convex in an upward direction (positive direction of the Z-axis (second direction)). In other words, the upper hyperboloid mirror <NUM> and the lower hyperboloid mirror <NUM> are arranged such that the vertex of the upper hyperboloid mirror <NUM> and the vertex of the lower hyperboloid mirror <NUM> face each other and the central axis Ox coincides with each other. Note that a Z direction in <FIG> is typically preferably a direction substantially coinciding with the gravity direction in a case where the stereo camera <NUM> is mounted on an automobile, but is not limited thereto. In the example of <FIG>, the upper hyperboloid mirror <NUM> is disposed on the upper side and the lower hyperboloid mirror <NUM> is disposed on the lower side, but the vertical positional relationship can be reversed.

The lower hyperboloid mirror <NUM> is further divided into an inner hyperboloid mirror 103A (inner mirror) and an outer hyperboloid mirror 103B (outer mirror). The inner hyperboloid mirror 103A is disposed at a position including the vertex of the hyperboloid, and the outer hyperboloid mirror 103B is disposed on the outer peripheral side of the outer edge of the inner hyperboloid mirror <NUM>. The inner hyperboloid mirror <NUM> and the outer hyperboloid mirror 103B have a common central axis Ox. The inner hyperboloid mirror 103A and the outer hyperboloid mirror 103B have different conic constants. Further, both have a step at the boundary, and the inner hyperboloid mirror 103A protrudes upward from the outer hyperboloid mirror 103B.

The upper hyperboloid mirror <NUM> does not have a reflective surface over the entire circumference (<NUM>°) of the hyperboloid surface, and is cut out in a fan shape (first fan shape) having an internal angle θ1 of <NUM>° or more and <NUM>° or less. Similarly, the lower hyperboloid mirror <NUM> does not have a reflective surface over the entire circumference (<NUM>°) of the hyperboloid surface, and is cut out in a fan shape (second fan shape) having an internal angle θ2 of <NUM>° or more and <NUM>° or less. The internal angles θ1 and θ2 are set to angles corresponding to a horizontal view angle of the stereo camera <NUM>.

Since the upper hyperboloid mirror <NUM> and the lower hyperboloid mirror <NUM> have such fan shapes, the resolution of the image sensor <NUM> can be improved, and as a result, the accuracy of the distance measurement of the stereo camera can be improved. This point will be described later. The fan shape described herein is not limited to a shape obtained by linearly cutting out a part of a circle along a radius thereof. For example, as illustrated in <FIG>, a shape in which a reflective surface in the vicinity of the vertex is cut out is also included in the fan shape described herein. In the following description, it is assumed that the internal angles θ1 and θ2 have the same value, but the internal angles θ1 and θ2 may have different values.

The stereo camera <NUM> reflects a plurality of light fluxes having different paths from the subject on a hyperboloid mirror (the upper hyperboloid mirror <NUM> and the lower hyperboloid mirror <NUM>) in a range of a predetermined horizontal angle of view, thereby making it possible to acquire information on the distance to the subject. Specifically, the first light R1 from the subject is reflected by the upper hyperboloid mirror <NUM>, then further reflected on the lower hyperboloid mirror <NUM> (the inner hyperboloid mirror 103A), and then incident on the image sensor <NUM> via the imaging optical system <NUM>. In addition, the second light R2 different from the first light R1 from the subject is reflected on the lower hyperboloid mirror <NUM> (the outer hyperboloid mirror 103B) and is incident on the image sensor <NUM> via the imaging optical system <NUM>. In this manner, the first light R1 and the second light R2, which pass through different optical paths from the subject and reach the hyperboloid mirror, are incident on different portions on the image sensor <NUM> from different directions, and thereby the three-dimensional position and distance of the subject can be calculated.

The imaging optical system <NUM> is configured to include a combination of one or more lenses, and can be preferably disposed on a straight line connecting the vertexes of the upper hyperboloid mirror <NUM> and the lower hyperboloid mirror <NUM>.

The image sensor <NUM> includes, for example, a complementary metal oxide semiconductor (CMOS) sensor or a charge coupled device (CCD) sensor, and is configured to generate an image signal based on light received via the imaging optical system <NUM> and output the image signal to the drive control unit <NUM> under the control of the drive control unit <NUM>.

The drive control unit <NUM> supplies the image signal output from the image sensor <NUM> to the display <NUM>. The display <NUM> generates three-dimensional position information of a subject <NUM> based on the supplied output image. Specifically, the drive control unit <NUM> calculates the three-dimensional position or distance of the subject by analyzing the image signal based on the first light R1 and the second light R2. Note that the image itself received by the image sensor <NUM> may be displayed on the display <NUM>, or a three-dimensional image of the subject as a result of performing predetermined image processing may be displayed. The three-dimensional image information may be displayed on the display <NUM>, may be supplied to an electronic control unit (ECU) of an automobile on which the stereo camera <NUM> is mounted, or may be transmitted to an external server via a communication control unit (not illustrated).

Next, the upper hyperboloid mirror <NUM> and the lower hyperboloid mirror <NUM> (inner hyperboloid mirror 103A and outer hyperboloid mirror 103B) will be described in more detail with reference to <FIG>.

The hyperboloid of each of the upper hyperboloid mirror <NUM>, the inner hyperboloid mirror 103A, and the outer hyperboloid mirror 103B can be defined by a quadratic surface expressed by the following [Mathematical Formula <NUM>]. When the conic constant κ is smaller than -<NUM>, the quadratic surface becomes a hyperboloid. The absolute value of the conic constant κ of the inner hyperboloid mirror 103A is set to a value larger than the absolute value of the conic constant κ of the outer hyperboloid mirror 103B.

Here, z(r) in [Mathematical Formula <NUM>] is the sag amount of the surface in the optical axis direction with the vertex on the optical axis as an origin. c represents curvature on the optical axis (on-axis curvature), and r represents a radial coordinate from the optical axis.

In general, a hyperboloid has two focal points, and the coordinates thereof are expressed by the following equation [Mathematical Formula <NUM>] with respect to a surface vertex.

Note that the coordinates of the focal point on the inner side of the hyperboloid are represented by f when ± of [Mathematical Formula <NUM>] is +. Furthermore, in [Mathematical Formula <NUM>], the coordinates of the focal point on the outer side of the hyperboloid are represented by f in a case where ± is -. Hereinafter, a focal point on the inner side of the hyperboloid is referred to as a "first focal point", and a focal point on the outer side of the hyperboloid is referred to as a "second focal point".

The upper hyperboloid mirror <NUM> and the inner hyperboloid mirror 103A are arranged such that a second focal point FPu2 of the upper hyperboloid mirror <NUM> substantially coincides with a first focal point FPdi1 of the inner hyperboloid mirror 103A. Note that a first focal point FPdo1 of the outer hyperboloid mirror 103B does not need to substantially coincide with the second focal point FPu2 and the first focal point FPdi1, but the outer hyperboloid mirror 103B is preferably disposed so as to be in the vicinity thereof.

The inner hyperboloid mirror 103A and the outer hyperboloid mirror 103B are arranged such that a second focal point FPdo2 of the outer hyperboloid mirror 103B substantially coincides with a second focal point FPdi2 of the inner hyperboloid mirror 103A. The imaging optical system <NUM> is disposed at the positions of the second focal point FPdo2 and the second focal point FPdi2. Note that a first focal point FPu1 of the upper hyperboloid mirror <NUM> may substantially coincide with the positions of the second focal point FPdo2 and the second focal point FPdi2, but is preferably located in the vicinity thereof, specifically, above the positions of the second focal point FPdo2 and the second focal point FPdi2.

Since the upper hyperboloid mirror <NUM> and the lower hyperboloid mirror <NUM> have the above positional relationship, the first light R1 directed from the subject to the first focal point FPu1 is reflected by the upper hyperboloid mirror <NUM> and then directed to the second focal point FPu2. Since the second focal point FPu2 and the first focal point FPdi1 substantially coincide with each other, the light reflected by the inner hyperboloid mirror 103A is condensed toward the second focal point FPdi2 (second focal point FPdo2). This light is incident on the image sensor <NUM> via the imaging optical system <NUM>.

The second light R2 directed from the subject to the first focal point FPdo1 is reflected by the outer hyperboloid mirror 103B and then directed to the second focal point FPdo2 (second focal point FPdi2). The second light R2 is incident on the imaging optical system <NUM> at an incident angle different from that of the first light R1, and thereby the second light is incident on a light-receiving surface of the image sensor <NUM> at a position different from that of the first light R1.

As described above, the image sensor <NUM> captures the image of the subject viewed from the first focal point FPdo1 based on the first light R1, and projects the image of the subject viewed from the first focal point FPdo1 based on the second light R2, and these images are used as output images. As described above, the absolute value of the conic constant κ of the inner hyperboloid mirror 103A is set to a value larger than the absolute value of the conic constant κ of the outer hyperboloid mirror 103B. As a result, in the output image of the image sensor <NUM>, the sizes of the image of the subject viewed from the first focal point (upper viewpoint) FPu1 and the image of the subject viewed from the first focal point FPdo1 are aligned. Since the sizes of the images are aligned, the resolutions of both images can be matched, and the accuracy of the parallax matching processing can be improved.

Next, a positional relationship between the imaging optical system <NUM> and the image sensor <NUM> will be described with reference to <FIG>. The optical axis Ox2 of the imaging optical system <NUM> is disposed so as to coincide with the central axes Ox of the upper hyperboloid mirror <NUM> and the lower hyperboloid mirror <NUM>. On the other hand, the center position CP1 of the light-receiving surface Pi of the image sensor <NUM> is shifted from the optical axis Ox2. As a result, the light (fan-shaped image) reflected from the fan-shaped upper hyperboloid mirror <NUM> and the lower hyperboloid mirror <NUM> can be received by the wider light-receiving surface of the image sensor <NUM>, and as a result, the resolution of the stereo camera <NUM> can be improved. This will be described with reference to <FIG>.

<FIG> is an example of an image based on the first light R1 and the second light R2 formed on the light-receiving surface Pi of the image sensor <NUM> in a case where the center position CP1 substantially coincides with the optical axis Ox2. In this case, both a fan-shaped image Im1 based on the first light R1 and a fan-shaped image Im2 based on the second light R2 are formed mainly on one side (upper side in <FIG>) of the center position CP1 and are not formed on the other side. For this reason, the number of pixels contributing to the analysis of the images Im1 and Im2 decreases, and a sufficient resolution cannot be obtained.

On the other hand, <FIG> is an example of an image based on the first light R1 and the second light R2 formed on the light-receiving surface Pi of the image sensor <NUM> in a case where the center position CP1 is shifted from the optical axis Ox2. Unlike the case of <FIG>, the fan-shaped images Im1 and Im2 can be received by the wide light-receiving surface of the image sensor <NUM> on both sides rather than one side of the center position CP1. By appropriately setting the magnification of the imaging optical system <NUM> and a shift amount (<FIG>) of the image sensor <NUM>, the images Im1 and Im2 can be maximized on the light-receiving surface of the image sensor <NUM>, and thereby the resolution of the stereo camera <NUM> can be improved.

<FIG> and <FIG> illustrate an example of a relationship of between an aspect ratio AP of a light-receiving surface Pi of the image sensor <NUM>, an internal angle θ (θ1, θ2) of the upper hyperboloid mirror <NUM> and the lower hyperboloid mirror <NUM>, an angle α (hereinafter, referred to as "axial inclination α") between a center line of the fan-shaped images Im1 and Im2 on the image sensor <NUM> and a short side Lx of the light-receiving surface of the image sensor <NUM>, and a light receiving state of the light-receiving surface Pi of the image sensor <NUM>. Here, the internal angle θ1 = θ2 is assumed.

<FIG> illustrates an example of a simulation result of the light receiving state of the image sensor <NUM> in a case where the aspect ratio AP is set to <NUM>, the internal angle θ is set to <NUM>°, the axial inclination α is set to <NUM>°, and the magnification of the imaging optical system <NUM> is set to the maximum. <FIG> also illustrate examples of simulation results of the light receiving state of the image sensor <NUM> in a case where the numerical values illustrated in <FIG> are set. <FIG> illustrates an example in a case where the aspect ratio AP is set to <NUM> (<NUM>:<NUM>) and <NUM> (<NUM>:<NUM>).

As is clear from <FIG> and <FIG>, the resolution of the stereo camera <NUM> can be further improved by setting the optimum axial inclination α according to the given aspect ratio AP and internal angle θ. The image sensor <NUM> and the imaging optical system <NUM> are disposed such that center lines of the fan-shaped images Im1 and Im2 projected onto light-receiving surface Pi of the image sensor <NUM> are non-parallel to the short side Lx of the light-receiving surface Pi of the image sensor <NUM>. As a result of simulations by the applicant including <FIG> and <FIG>, it has been found that when the aspect ratio AP is <NUM>, the axial inclination α is preferably <NUM>° or more if the internal angle θ is <NUM>° or less. In addition, it has been found that when the aspect ratio AP is <NUM> (<NUM>:<NUM>), the axial inclination α is preferably <NUM>° or more if the internal angle θ is <NUM>° or less.

<FIG> is a graph illustrating a combination of an internal angle θ and an axial inclination α with which the resolution in the image sensor <NUM> can be maximized for each aspect ratio AP of the image sensor <NUM>. As illustrated in <FIG>, when the aspect ratio AP is <NUM>, regardless of the magnitude of the internal angle θ, the axial inclination α is optimally <NUM> degrees. However, in a case where the aspect ratio AP is less than <NUM>, it is preferable that the larger the internal angle θ, the smaller the axial inclination α.

As described above, it is preferable that the inner hyperboloid mirror 103A and the outer hyperboloid mirror 103B have a step at the boundary, and the inner hyperboloid mirror 103A protrudes upward as compared with the outer hyperboloid mirror 103B. <FIG> illustrates an example of the light receiving state of the first light R1 and the second light R2 when the inner hyperboloid mirror 103A protrudes upward as compared with the outer hyperboloid mirror 103B as in the present embodiment. On the other hand, <FIG> illustrates an example of the light receiving state of the first light R1 and the second light R2 when there is no step between the inner hyperboloid mirror 103A and the outer hyperboloid mirror 103B unlike the present embodiment.

In the case of <FIG>, the distance between the first focal point FPu1 and the first focal point FPdo1 is large. With this, in the image sensor <NUM>, the distance between the light receiving positions of the first light R1 and the second light R2 is also large, and the angle of view that can be stereoscopically viewed is small. In this regard, in the first embodiment (<FIG>), the distance between the light receiving positions of the first light R1 and the second light R2 can be reduced, and the angle of view that can be stereoscopically viewed can be increased. Alternatively, the resolution of the image sensor <NUM> can be improved, and the detection accuracy of the distance to the subject can be improved.

In the first embodiment described above, hyperboloid mirrors arranged to face each other are used as a pair of mirrors that reflects light from the subject. However, the pair of mirrors is not limited to the hyperboloid mirrors as illustrated in the drawing, and a spherical mirror, a paraboloid mirror, an elliptical mirror, an aspherical mirror, or the like arranged opposite to each other can be adopted instead of the hyperboloid mirrors.

As described above, in the stereo camera <NUM> of the first embodiment, the upper hyperboloid mirror <NUM> and the lower hyperboloid mirror <NUM> constituting the hyperboloid mirror are formed in a fan shape having the internal angle θ of <NUM>° or less. Furthermore, the center position of the image sensor <NUM> is set to a position shifted with respect to the optical axis of the imaging optical system so that the image from the fan-shaped hyperboloid mirror can be received to the maximum. With this, it is possible to provide a compact stereo camera capable of performing high-accuracy and wide-angle sensing.

Next, a stereo camera <NUM> according to a second embodiment will be described with reference to <FIG>. In <FIG>, the same components as those in <FIG> are denoted by the same reference numerals as those in <FIG>, and thus redundant description will be omitted.

In the second embodiment, a reflection mirror <NUM> is provided between the imaging optical system <NUM> and the lower hyperboloid mirror <NUM>. Furthermore, the arrangement position of the imaging optical system <NUM> is also the reflection direction of the reflection mirror <NUM>, which is different from the first embodiment in which the imaging optical system <NUM> is disposed in the vicinity of the vertex of the hyperboloid of the upper hyperboloid mirror.

According to this configuration, since the incident light to the imaging optical system <NUM> can be folded back by the reflection mirror <NUM>, the imaging optical system <NUM> and the image sensor <NUM> can be housed within the range of the height of the hyperboloid mirrors (the upper hyperboloid mirror <NUM> and the lower hyperboloid mirror <NUM>), so that the size of the stereo camera <NUM> can be reduced.

Next, a stereo camera-integrated headlight unit according to a third embodiment will be described with reference to <FIG> is a perspective view illustrating a configuration example of a stereo camera-integrated headlight unit <NUM> according to the third embodiment.

The stereo camera-integrated headlight unit <NUM> is obtained by integrating a stereo camera <NUM> similar to that of the above-described embodiment into a housing <NUM> of a headlight of an automobile. The stereo camera-integrated headlight unit <NUM> may incorporate a turn lamp, a positioning lamp, or the like.

Note that a configuration example of the stereo camera-integrated headlight unit <NUM> illustrated in <FIG> is assumed to be attached to left and right front ends of an automobile. On the left and right rear ends of the automobile, a tail lamp unit and a stop lamp unit similar to those in <FIG> can be provided.

The stereo camera-integrated headlight unit <NUM> includes, as an example, an LED light source (not illustrated), a reflector <NUM> that reflects and focuses white light emitted from the LED light source toward the front of the automobile, a shade <NUM> that blocks the emitted light in order to form a cut-off line of a low beam, and a headlight structure (light irradiation unit) including a lens <NUM> that forms an image of an edge blocked by the shade <NUM> at a distance. Note that, in a case of emitting a high beam, for example, the shade <NUM> can be made movable and shifted from the optical path.

According to the stereo camera-integrated headlight unit <NUM> in which the stereo camera <NUM> and the headlight are integrated, an installation area can be reduced as compared with a case where the stereo camera <NUM> is disposed at the left and right front ends of the automobile separately from the headlight. Furthermore, handling is facilitated, and assembly and positioning with respect to the vehicle body can be easily performed. Furthermore, in the configuration example of <FIG>, in the headlight unit <NUM>, the stereo camera <NUM> is disposed below the headlight structure (<NUM> to <NUM>). According to this disposition, since the stereo camera <NUM> can be disposed without blocking the light emitted from the headlight structure, so-called nose view can be easily realized in the automated driving vehicle.

Claim 1:
A stereo camera comprising:
a first mirror (<NUM>) that has a first reflective surface which is a curved surface convex in a first direction, has a first vertex, and has a first fan shape;
a second mirror (<NUM>) that has a second reflective surface convex in a second direction opposite to the first direction, has a second vertex facing the first vertex, and has a second fan shape;
an imaging optical system (<NUM>) that forms an image of first light (R1) emitted from a subject, reflected on the first reflective surface, and then further reflected on the second reflective surface, and second light (R2) emitted from the subject and reflected on the second reflective surface; and
an image sensor (<NUM>) that receives the first light (R1) and the second light (R2) via the imaging optical system (<NUM>),
wherein
the second mirror (<NUM>) includes an inner mirror (103A) and an outer mirror (103B) located on an outer peripheral side of the inner mirror (103A),
the first mirror (<NUM>) is a hyperboloid mirror,
the inner mirror (103A) is a hyperboloid mirror,
the outer mirror (103B) is a hyperboloid mirror, the outer mirror (103B) has a conic constant different from a conic constant of the inner mirror (103A),
the image sensor (<NUM>) is disposed at a position where a center position of the image sensor (<NUM>) is shifted from an optical axis of the imaging optical system (<NUM>), and
the image sensor (<NUM>) and the imaging optical system (<NUM>) are disposed such that a center line of a fan-shaped image projected on a light-receiving surface (Pi) of the image sensor (<NUM>) is non-parallel to a short side of the light-receiving surface (Pi) of the image sensor (<NUM>).