Method for detecting a mark made on a ground, driver assistance device and motor vehicle

The invention relates to a method for using a driver assistance device of a motor vehicle (1) to detect a mark (7) made on a ground (6) by providing a two-dimensional image by means of an imaging sensor (3) of the driver assistance device (2); by identifying a potential mark (7″) on the basis of the two-dimensional image (11); and by performing a distance measurement by means of a distance sensor (4) of the driver assistance device and also taking a result (17) of the distance measurement as a basis for determining whether the potential mark (7″) is the mark (7) made on the ground (6) or a mark (14) on an object (13) that is on the ground (6).

The invention relates to a method for using a driver assistance device of a motor vehicle to detect a marking made on a ground. The invention furthermore relates to a driver assistance device that is designed to perform such a method, and to a motor vehicle having such a driver assistance device.

The concern in the present case is directed particularly towards parking assistance systems that are designed to perform parking manoeuvres for a motor vehicle. In this regard, a distinction is drawn between what are known as semiautonomous (semiautomatic) systems, on the one hand, and fully automatic systems, on the other. A common feature of all systems in this case is that a potential parking space is detected and a size of the parking space is recorded and then compared with a stored minimum size. If the recorded size of the detected parking space is at least equal to the minimum size, then the possibility of the parking manoeuvre is signalled to the driver. The system then ascertains the current relative position of the motor vehicle with respect to the parking space and computes a parking path (parking trajectory) along which the motor vehicle can be parked in the parking space without collision. In the case of semiautonomous parking assistance systems, the driver is assigned the tasks of accelerating and braking. In the case of semiautonomous parking assistance systems, longitudinal guidance of the motor vehicle is thus controlled by the driver himself, while transverse guidance is controlled by the parking assistance system itself by virtue of appropriate control signals being output to a steering apparatus of the motor vehicle. In the case of fully automatic parking assistance systems, on the other hand, both longitudinal guidance and transverse guidance are performed automatically by the parking assistance system by virtue of appropriate control signals being output both to the steering apparatus and to a drive train and a brake system of the motor vehicle. In the case of fully automatic systems, the driver merely needs to approve the parking manoeuvre and can also interrupt it at any time.

For the purpose of recording parking spaces, ultrasonic sensors are usually used that are fitted to the respective lateral edges in the front region of the motor vehicle. As the motor vehicle travels past a potential parking space, the ultrasonic sensor continuously measures the lateral distance of the motor vehicle from obstacles or objects that are external to the vehicle. On the basis of the profile of the measured values from the ultrasonic sensor and taking account of the distance covered, it is then possible to determine the size of the parking space in the longitudinal direction of the motor vehicle or in the longitudinal direction of the road and also the relative position of the parking space with respect to the motor vehicle. This applies both to parallel parking spaces, the longitudinal axis of which is oriented essentially parallel to the road, and to transverse parking spaces, the longitudinal axis of which runs perpendicular or at an acute angle with respect to the road. Such a detection method is known from the document EP 0 305 907 B1, for example.

It is furthermore already prior art to detect parking spaces by using both the sensor data from ultrasonic sensors and image data from a camera and combining them with one another. Such sensor fusion is known from the document DE 10 2007 047 362 A1, for example.

A method for providing an image presentation on a display using image data from a camera and sensor data from multiple ultrasonic sensors is known from the document DE 11 2010 005 572 T5.

In parking assistance systems today, which are designed to detect parking spaces on the basis of image data from a camera and on the basis of sensor data from an ultrasonic sensor, a parking space is usually detected by means of an ultrasonic sensor first of all. When a parking space is detected on the basis of sensor data from the ultrasonic sensor, the image data are additionally evaluated and the originally detected parking space is corrected if need be. Such correction is necessary, for example, when a vehicle bounding the parking space is parked obliquely, i.e. at an acute angle with respect to the longitudinal direction of the parking space. In order to be able to prevent the ego motor vehicle from also being parked obliquely in the detected parking space by the parking assistance system in such cases, recognition of the parking space markings by means of a camera is necessary. Furthermore, image data can also be used to detect parking spaces that have no vehicles parked next to them and that are therefore bounded exclusively by appropriate parking space markings on the ground. Line recognition works particularly reliably particularly when the parking space is bounded by T-shaped markings.

Simple parking space markings that do not have the T shape can be detected only inadequately, on the other hand. Such markings can easily be confused with shadows or with trims on other vehicles.

It is an object of the invention to demonstrate a solution to how a method of the type cited at the outset allows the driver assistance device to be used to detect markings made on a ground particularly reliably.

The invention achieves this object by means of a method, by means of a driver assistance device and by means of a motor vehicle having the features according to the respective independent patent claims. Advantageous embodiments of the invention are the subject matter of the dependent patent claims, the description and the figures.

A method according to the invention allows a driver assistance device of a motor vehicle that is on a ground, for example on a road, to be used to detect a marking made on the ground. An imaging sensor of the driver assistance device is used to provide a two-dimensional image of vehicle surroundings, and a potential marking is identified on the basis of the two-dimensional image. A distance measurement is performed by means of a distance sensor of the driver assistance device, which distance is separate from the imaging sensor. A result of the distance measurement is then taken as a basis for determining whether the potential marking detected on the basis of the image is the marking made on the ground or else a marking on an object that is on the ground.

According to the invention, the potential marking identified on the basis of the two-dimensional image is accordingly plausibilized on the basis of sensor data that are provided by a distance sensor that can also detect three-dimensional objects. Since the two-dimensional image contains no depth information and the potential marking is detected by means of a ground projection of the image, solely the use of the two-dimensional image does not allow it to be established whether the potential marking is actually a marking made on the ground or else a marking situated above the ground on a three-dimensional object, for example a trim of elongate design on a vehicle or the like. If the distance measurement is performed by means of the distance sensor, then it is possible for the potential marking to be plausibilized, and it is possible to reliably establish whether the potential marking is actually a marking made on the ground or a marking on the object. The detection of ground markings can therefore take place particularly reliably.

The marking detected in the case of the method is preferably a longitudinal marking, i.e. an elongate, linear marking. By way of example, a marking may be a parking space marking that bounds a parking space. Additionally or alternatively, the marking may also be a longitudinal road marking that defines a lane on a road.

Preferably, the driver assistance device determines a position of the potential marking relative to the motor vehicle on the basis of the two-dimensional image, and plausibilizes the potential marking by comparing said position with the result of the distance measurement. The relative position of the potential marking is preferably determined by means of a ground projection of the potential marking onto the ground, specifically along a straight line connecting the imaging sensor to the potential marking. Since the two-dimensional image contains no depth information, the relative position of the potential marking can be determined only by projecting this marking onto a plane of the ground. If this relative position of the potential marking is known, then it can be compared with the result of the distance measurement and hence plausibilized. This makes it possible to rule out errors in the detection of markings.

In this connection, there may be the particular provision that if the distance measurement involves a distance being measured that is shorter than a distance of the potential marking from the motor vehicle, then the potential marking is interpreted as the marking on the object and hence not a genuine ground marking. Since, in order to be able to take the two-dimensional image as a basis for determining the position of the potential marking, the marking is projected onto the ground plane, this marking is possibly at a greater distance from the motor vehicle than the distance measured by the distance sensor. Since it is not plausible for an actual ground marking to be able to be detected behind a three-dimensional object, this must be a misdetection. If an object is thus detected in front of the potential marking, this potential marking is interpreted as a line above the ground and can therefore be rejected.

If, on the other hand, the distance measurement involves no object being detected or a distance being measured that is longer than a distance of the potential marking from the motor vehicle, then the potential marking is preferably interpreted as the marking made on the ground and hence a genuine marking. In this case, detection of the ground marking is confirmed, as a result of which it can be interpreted as a parking space marking or else a longitudinal road marking, for example. The detection is therefore plausible.

In respect of the recording of the two-dimensional image, a wide variety of embodiments may be provided: first, the provision of two-dimensional image data can involve the use of a camera, particularly a colour-sensitive camera, such as a CMOS camera or a CCD camera, for example. Such a camera is an image recording device that can detect light in the visible spectral range and hence can provide images of the vehicle surroundings. Although image data from a camera can be taken as a basis for detecting markings made on the ground, the image data in a single frame contain no depth information. For this reason, the position of a detected marking can be ascertained only by means of the projection onto the ground plane. Specifically when using such a camera, the plausibilization according to the invention by means of a distance sensor is found to be particularly advantageous.

Additionally or alternatively, the imaging sensor used can also be a laser scanner.

The distance measurement is preferably effected using an ultrasonic sensor. Additionally or alternatively, the distance sensor used can also be a laser scanner and/or a radar sensor and/or a camera providing three-dimensional image data. Such a camera may be a stereo camera and/or a camera in which three dimensional image information is provided on the basis of multiple images that are recorded during the journey of the motor vehicle, for example. In this case, the SLAM (simultaneous localization and mapping) method can be used, for example.

As already explained, the driver assistance device may be in the form of a parking assistance system by means of which parking spaces are detected and the motor vehicle is parked at least semiautonomously. As a marking, the parking assistance system can then detect a parking space marking that bounds a parking space. On the basis of the detected parking space marking, the system can then detect a parking space and/or stipulate a target position for the motor vehicle in a parking space. Hence, at least semiautonomous parking manoeuvres can be performed particularly reliably, since the precise orientation of the parking space can be ascertained on the basis of the detected parking space marking.

Additionally or alternatively, the driver assistance device may also be designed to recognize a lane on a road. The marking detected is then a longitudinal road marking. In this embodiment, the driver assistance device may be in the form of a lane departure warning system, for example.

The invention furthermore relates to a driver assistance device for a motor vehicle that is designed to detect a marking made on a ground and comprises an imaging sensor for providing a two-dimensional image, an evaluation device for identifying a potential marking on the basis of the two-dimensional image and a distance sensor for performing a distance measurement. The evaluation device is designed to take a result of the distance measurement as a basis for determining whether the potential marking is the marking made on the ground or a marking on an object that is on the ground.

A motor vehicle according to the invention, particularly a car, comprises a driver assistance device according to the invention.

The preferred embodiments presented with reference to the method according to the invention, and the advantages of said embodiments, apply accordingly to the driver assistance device according to the invention and to the motor vehicle according to the invention.

Further features of the invention will emerge from the claims, the figures and the description of the figures. All features and combinations of features cited in the description above and the features and combinations of features that are cited in the description of the figures below and/or in the figures alone can be used not only in the respectively indicated combination but also in other combinations or else on their own.

A motor vehicle1as shown inFIG. 1is a car having a driver assistance device2according to an embodiment of the invention, for example. In the exemplary embodiment, the driver assistance device2is in the form of a parking assistance system that is designed to detect parking spaces and to perform semiautomatic or fully automatic parking manoeuvres. The parking assistance system can automatically compute a suitable parking path along which the motor vehicle1can then be guided automatically or semiautonomously into a previously detected parking space. In the case of fully automatic parking assistance systems, the driver assistance device2undertakes both longitudinal guidance and transverse guidance for the motor vehicle1, whereas in the case of semiautomatic or semiautonomous systems, the driver assistance device2controls only the transverse guidance and hence the steering automatically, while the driver needs to accelerate and brake himself.

The driver assistance device2may additionally or alternatively also be in the form of a lane departure warning system.

The driver assistance device2comprises an imaging sensor3that is in the form of a camera in the exemplary embodiment. The driver assistance device2also includes a multiplicity of distance sensors4, which are ultrasonic sensors, for example. The distance sensors4may be in a distributed arrangement on the front bumper of the motor vehicle1, for example. Corresponding distance sensors4may also be arranged on the rear bumper. At least one distance sensor4is arranged on the respective lateral face of the motor vehicle1.

The driver assistance device2furthermore comprises an electronic evaluation device5that is designed to detect markings7made on a ground6—in this case a road. When the markings7are detected, the detected markings7can be taken as a basis for detecting a parking space8and/or stipulating a target position for the motor vehicle1within the parking space8. Detection of the markings7allows determination of the orientation of the parking space8in situations in which the parking space8is laterally bounded by obliquely parked vehicles9,10. Using the distance sensors4only, precise recording of the parking space orientation is not possible, or possible only to a limited degree, in such cases.

The markings7are detected using image data from the imaging sensor3. An exemplary image11of the parking space8′ or of the markings7′ is shown schematically inFIG. 2, where12denotes a horizon. In this case, the image11corresponds to an image of the vehicle surroundings that is provided by the camera. In order to detect the markings7′, what is known as a gradient image or edge image can be produced from the image from the camera and is then taken as a basis for identifying the markings7′.

If no further object is situated between the motor vehicle1and the markings7, then the markings7can be detected without any problem, in principle, on the basis of image data from the sensor3. Difficulties can arise in the prior art when—as shown inFIG. 3—there is an object13on the ground6, for example in the region of an actual marking7. If the object13is a dark vehicle with a light, elongate chrome strip14, for example, then the sensor3provides an image11as shown schematically inFIG. 4. This depicts an object13in the form of a black vehicle with an elongate chrome strip14, the object13standing on a dark ground6′. Without additional plausibilization, the evaluation device5would interpret the chrome strip14as a ground marking.

In order to prevent such erroneous interpretations, the evaluation device5performs additional plausibilization for the potential markings detected on the basis of the image11. Referring again toFIG. 3, the imaging sensor3provides an image that depicts the object13—for example with a cited chrome strip14or another elongate marking. This chrome strip14is interpreted by the evaluation device5as a potential marking7″ that is in a relative position with respect to the motor vehicle1that is obtained by means of a ground projection15along a straight line16connecting the sensor3to the chrome strip14. The reason is that the imaging sensor3can only provide a two-dimensional image11of the surroundings that contains no depth information. In order to plausiblize or confirm this potential marking7″, a distance measurement is now performed using the distance sensor4, which measures a distance17from the object13. The evaluation device5then checks whether the measured distance17is shorter or longer than a distance of the potential marking7″ from the motor vehicle1. In other words, the evaluation device5checks whether or not there is an object13in front of the potential marking7″—i.e. between the motor vehicle1and the potential marking7″. If the object13is detected in front of the potential marking7″, the potential marking7″ is interpreted by the evaluation device5as the chrome strip14and hence as a marking on the object13. If no object13is detected in the region between the motor vehicle1and the potential marking7″, on the other hand, then the potential marking7″ is interpreted as a marking7that is actually made on the ground6and whose position corresponds to the position of the potential marking7″ inFIG. 3.