Patent ID: 12211291

DESCRIPTION OF EMBODIMENTS

An apparatus for detecting a lane boundary as well as a method and a computer program therefor executed by the apparatus will now be described with reference to the attached drawings. The apparatus inputs an image representing the surroundings of a vehicle into a classifier to identify the types of objects represented in respective pixels of the image. The image is obtained by a camera mounted on the vehicle, and the classifier has been trained to identify the type of object represented in each pixel. For each of pixel lines in a direction crossing the lengthwise direction of the travel lane in the image (hereafter simply a “direction crossing the travel lane”), the apparatus then determines whether the position corresponding to a pixel group including a predetermined number of contiguous pixels is inside the travel lane in order along a scanning direction from one end to the other end of the pixel line, depending on the order of the types of objects represented in the pixel group and the result of determination whether the position corresponding to the immediately preceding pixel group with respect to the scanning direction is inside the travel lane. The apparatus thereby detects the positions of the left and right boundaries of the travel lane viewed from the vehicle. In this way, the apparatus detects the positions of the boundaries of the travel lane by one scan per pixel line, aiming to reduce the computational burden. Additionally, the apparatus only has to scan along a particular direction (e.g., from left to right), and thus matches the order of pixels of the image stored in a memory with the scanning direction, aiming to improve the efficiency of memory access.

The following describes an example in which the apparatus for detecting a lane boundary is applied to a vehicle control system. In this example, the apparatus executes a boundary detection process on an image obtained by a camera mounted on a vehicle to detect the boundaries of the travel lane, and uses the result of detection for autonomous driving control of the vehicle.

FIG.1schematically illustrates the configuration of a vehicle control system equipped with the apparatus for detecting a lane boundary.FIG.2illustrates the hardware configuration of an electronic control unit, which is an embodiment of the apparatus for detecting a lane boundary. In the present embodiment, the vehicle control system1, which is mounted on a vehicle10and controls the vehicle10, includes a camera2for capturing the road surface around the vehicle10, and an electronic control unit (ECU)3, which is an example of the apparatus for detecting a lane boundary. The camera2is connected to the ECU3via an in-vehicle network conforming to a standard, such as a controller area network, so that they can communicate with each other. The vehicle control system1may further include a storage device (not illustrated) that stores a map used for autonomous driving control of the vehicle10. The vehicle control system1may further include a distance sensor (not illustrated), such as a LiDAR sensor or radar; a GPS receiver or another receiver (not illustrated) for determining the position of the vehicle10in conformity with a satellite positioning system; a wireless communication terminal (not illustrated) for wireless communication with another device; and a navigation device (not illustrated) for searching for a planned travel route of the vehicle10.

The camera2, which is an example of the image capturing unit that generates an image representing a region around the vehicle10, includes a two-dimensional detector constructed from an array of optoelectronic transducers, such as CCD or C-MOS, having sensitivity to visible or infrared light and a focusing optical system that forms an image of a target region on the two-dimensional detector. The camera2is mounted, for example, in the interior of the vehicle10so as to be oriented to the front of the vehicle10. The camera2captures a region including the road surface ahead of the vehicle10every predetermined capturing period (e.g., 1/30 to 1/10 seconds), and generates images representing this region. The images obtained by the camera2may be color or grayscale images. The vehicle control system1may include multiple cameras2taking pictures in different orientations or having different angles of view.

Whenever generating an image, the camera2outputs the generated image to the ECU3via the in-vehicle network.

The ECU3controls the vehicle10. In the present embodiment, the ECU3detects the boundaries of the travel lane from time-series images obtained by the camera2, and controls the vehicle10so that it will travel along the travel lane identified by the detected boundaries. To achieve this, the ECU3includes a communication interface21, a memory22, and a processor23.

The communication interface21, which is an example of a communication unit, includes an interface circuit for connecting the ECU3to the in-vehicle network. In other words, the communication interface21is connected to the camera2via the in-vehicle network. Whenever receiving an image from the camera2, the communication interface21passes the received image to the processor23. Additionally, the communication interface21passes a map read from the storage device and received via the in-vehicle network, positioning information received from the GPS receiver, and other information to the processor23.

The memory22, which is an example of a storage unit, includes, for example, volatile and nonvolatile semiconductor memories. The memory22stores a computer program for implementing various processes executed by the processor23of the ECU3as well as various types of data used in the boundary detection process, such as images received from the camera2and various parameters for specifying a classifier used in the boundary detection process. Additionally, the memory22stores the results of computation obtained during the boundary detection process.

The processor23, which is an example of a control unit, includes one or more central processing units (CPUs) and a peripheral circuit thereof. The processor23may further include another operating circuit, such as a logic-arithmetic unit, an arithmetic unit, or a graphics processing unit. Whenever receiving an image from the camera2during travel of the vehicle10, the processor23executes a vehicle control process including the boundary detection process on the received image. The processor23controls the vehicle10to automatically drive the vehicle10or assist a driver of the vehicle10in driving, based on the detected boundaries of the travel lane.

FIG.3is a functional block diagram of the processor23of the ECU3, related to the vehicle control process including the boundary detection process. The processor23includes an identification unit31, a detection unit32, and a vehicle control unit33. These units included in the processor23are, for example, functional modules implemented by a computer program executed on the processor23, or may be dedicated operating circuits provided on the processor23. Of these units included in the processor23, the identification unit31and the detection unit32execute the boundary detection process.

Whenever an image is obtained from the camera2, the identification unit31inputs the image into a classifier that has been trained to identify the type of object represented in each pixel, thereby identifying the types of objects represented in respective pixels of the image.

In the present embodiment, the types of objects identified by the classifier are used for detecting the boundaries of the travel lane represented in the image; thus the types of objects to be identified at least include types indicating the inside of the travel lane and types indicating the outside of the travel lane. In the present embodiment, the types of objects indicating the inside of the travel lane include the travel lane itself. The types of objects indicating the outside of the travel lane include a stationary object, a road surface different from the travel lane (hereafter, a “different road surface”), and a lane line. Mark portions and portions between the marks of a broken lane line on a road surface (the latter portions will hereafter be referred to as “non-mark portions”) may be separately identified. In the following description, mark portions and non-mark portions of a lane line will be collectively referred to as a lane line, unless otherwise specified. The types of objects to be identified by the classifier may further include a moving object or a different marking provided on a road surface along a lane line, such as a guiding lane marking or a deceleration marking.

For example, a neural network for semantic segmentation having a convolutional neural network architecture including multiple convolution layers, e.g., a fully convolutional network (FCN) or U-Net, is used as the classifier. The use of such a neural network as the classifier enables the identification unit31to relatively accurately identify the type of object represented in each pixel. As the classifier, one configured by a machine learning system different from a neural network, e.g., a classifier for semantic segmentation based on a random forest, may be used.

FIG.4illustrates an example of an image of the road surface ahead of the vehicle10and the result of identification of the types of objects in respective pixels of the image. In the example illustrated inFIG.4, the image400represents solid lane lines402and dotted guiding lane markings403on the left and right sides of a travel lane401.

Thus, in the result410of identification regarding the pixels of the image400, the pixels are classified into the travel lane401, the lane lines402, the different markings (guiding lane markings)403, a different road surface404, a stationary object405, and a moving object406. In this example, the lane opposite to the travel lane of the vehicle10is identified as the stationary object405rather than the different road surface404.

The identification unit31notifies the result of identification regarding the pixels to the detection unit32.

The detection unit32determines sets of pixels representing the left and right boundaries of the travel lane, based on the result of identification of objects in respective pixels. In the present embodiment, the detection unit32determines the edge of a lane line on the side of the travel lane as a boundary of the travel lane.

Suppose a process including, for each of pixel lines in a direction crossing the travel lane, setting a reference point inside the travel lane, scanning in a direction away from the reference point, and referring to the result of identification of objects in respective pixels to detect the position where a lane line first appears as a boundary of the travel lane. In this case, the computational burden will be heavy because each pixel line is scanned once to set a reference point and thereafter scanned again. Such a process involving multiple scans per pixel line to detect a boundary of the travel lane is not preferable. Additionally, if the scanning direction differs from the order of pixel values stored in the memory22or a cache memory of the processor23, the efficiency will be low in terms of memory access.

Thus, for each of pixel lines in a direction crossing the travel lane in the image, the detection unit32sets a scanning direction from one end to the other end of the pixel line. The detection unit32then detects the left and right boundaries of the travel lane by determining, for each pixel line, whether the position corresponding to a pixel group including a predetermined number of contiguous pixels is inside the travel lane in order along the scanning direction, depending on the order of the types of objects represented in the pixel group and the result of determination whether the position corresponding to the immediately preceding pixel group with respect to the scanning direction is inside the travel lane.

In the present embodiment, since the lengthwise direction of the travel lane is the vertical direction in the image, the scanning direction is set as the horizontal direction so as to cross the lengthwise direction of the travel lane. Additionally, individual pixel values of each horizontal pixel line in the image are sequentially stored from the left end to the right. Thus, to improve the efficiency of memory access, the detection unit32sets a start point of a scan at the left end of each horizontal pixel line and scans each pixel line from the start point to the right. In the present embodiment, the detection unit32scans pixel groups each including two horizontally adjacent pixels in each pixel line to find out the order of the types of objects represented in each pixel group.

FIG.5is a state transition diagram indicating how the position corresponding to a pixel of interest in a scan transitions, based on the order of the types of objects in each pixel group. In the present embodiment, the types of objects to be identified include seven types of objects: a travel lane501, a mark portion502of a lane line, a non-mark portion503of a lane line, a different marking504, a different road surface505, a moving object506, and a stationary object507, as described above. Thus, of 49 possible orders of the types of objects in each pixel group, 27 orders focused on at detection of a boundary of the travel lane are classified into 15 cases of cases 1 to 15. For example, case 1 indicates that the left and right pixels of the pixel group correspond to a lane line and a different marking, respectively; case 13 indicates that the types of objects are arranged opposite to those in case 1. Case 2 indicates that the left and right pixels of the pixel group correspond to a different marking and the travel lane, respectively; case 7 indicates that the types of objects are arranged opposite to those in case 2. Case 3 indicates that the left pixel of the pixel group corresponds to a lane line or a different road surface and the right pixel to the travel lane; case 8 indicates that the types of objects are arranged opposite to those in case 3. Additionally, case 4 indicates that the left and right pixels of the pixel group correspond to a moving object and the travel lane, respectively; case 9 indicates that the types of objects are arranged opposite to those in case 4. Additionally, case 6 indicates that the left and right pixels of the pixel group correspond to a stationary object and the travel lane, respectively; case 10 indicates that the types of objects are arranged opposite to those in case 6. Additionally, cases 5, 11, and 12 indicate that every pixel of the pixel group is the travel lane, a different marking, and a moving object, respectively. Additionally, case 14 indicates that the left pixel of the pixel group corresponds to a different marking and the right pixel to a different road surface, a moving object, or a stationary object. In case 15, the left pixel of the pixel group corresponds to a moving object whereas the right pixel corresponds to an object that is neither moving object nor the travel lane.

The state transition diagram500includes five states511to515. Of these, the state511indicates that a position of interest is outside the travel lane. The state512indicates that a position of interest is inside the travel lane. The state513indicates that a position of interest may be inside the travel lane and represents a moving object. Additionally, the state514indicates that a position of interest may be inside the travel lane and represents a different marking. The state515is an initial state of a scan, and indicates that a position of interest has not been identified whether it is inside or outside the travel lane. Each arrow between the states indicates a transition from the state at its origin to the state at its end; a number attached to each arrow indicates the case number (one of cases 1 to 15) of the pixel group to which the state transition is applied. The process associated with each case number is one to be executed together with the state transition when the types of objects in the pixel group of interest are arranged as indicated by the case number. Additionally, “right end” and “left end” mean the positions of the right and left boundary lines of the travel lane, respectively. Additionally, (k−1) and k indicate the positions corresponding to the left and right pixels of the pixel group of interest, respectively.

Upon starting a scan, the detection unit32sets the pixel group of interest so that the pixel of the left end, which is the start position of the scan, will be the position of interest and the left pixel of the pixel group of interest. The detection unit32then sets the state of the position of interest at one of the states511to514according to the type of object in the leftmost pixel. More specifically, when the type of object in the leftmost pixel is a stationary object, a different road surface, or a lane line, the detection unit32determines that the state of the position of interest is the state511, i.e., outside the travel lane. When the type of object in the leftmost pixel is the travel lane, the detection unit32determines that the state of the position of interest is the state512, i.e., inside the travel lane. When the type of the leftmost pixel is a moving object, the detection unit32determines that the state of the position of interest is the state513, which indicates that the position of interest may be inside the travel lane and represents a moving object. In other words, the detection unit32determines the position of interest as that of a candidate for the inside of the travel lane. When the type of the leftmost pixel is a different marking, the detection unit32determines that the state of the position of interest is the state514, which indicates that the position of interest may be inside the travel lane and represents a different marking. In this case also, the detection unit32determines the position of interest as that of a candidate for the inside of the travel lane.

Upon determining the state of the first position of interest, the detection unit32determines which of cases 1 to 15 the order of the types of objects in the pixel group of interest corresponds to. The detection unit32then refers to the state transition diagram500to change the state according to the order of the objects in the pixel group of interest in the state of the position of interest. For example, when the state of the position of interest is the state511, i.e., outside the travel lane, and the order of the types of objects in the pixel group of interest corresponds to case 3, the type of object in the left pixel of the pixel group of interest is a lane line or a different road surface, and that in the right pixel of this pixel group is the travel lane. Hence the detection unit32sets the position of the right pixel of the pixel group of interest as a valid position of the left boundary of the travel lane, and changes the state to the state512, i.e., inside the travel lane. When the state of the position of interest is the state511, i.e., outside the travel lane, and the order of the types of objects in the pixel group of interest corresponds to case 6, the types of objects in the left and right pixels of the pixel group of interest are a stationary object and the travel lane, respectively. Thus the stationary object adjoins the travel lane in the pixel group of interest without a lane line or a different road surface in between. Hence the detection unit32sets the position of the right pixel of the pixel group of interest as an invalid position of the left boundary of the travel lane, and changes the state to the state512, i.e., inside the travel lane. When the state of the position of interest is the state513, i.e., possibly inside the travel lane, and the order of the types of objects in the pixel group of interest corresponds to case 15 (the object in the right pixel is neither moving object nor the travel lane), the position of interest should be outside the travel lane rather than inside the travel lane. Hence the detection unit32changes the state to the state511, i.e., outside the travel lane, and discards the candidate for the travel lane set at the position of interest. Conversely, when the state of the position of interest is the state513, i.e., possibly inside the travel lane, and the order of the types of objects in the pixel group of interest corresponds to case 4 (the object in the right pixel is the travel lane), the position of interest should be inside the travel lane. Hence the detection unit32changes the state to the state512, i.e., inside the travel lane. Additionally, the detection unit32formally determines that the candidate for the travel lane set at the position of interest is inside the travel lane, and updates the assumed position of the right boundary of the travel lane to the position of the right pixel of the pixel group of interest. Similarly, when the state of the position of interest is the state514, i.e., possibly inside the travel lane, and the order of the types of objects in the pixel group of interest corresponds to case 13 or 14, the position of interest should be outside the travel lane rather than inside the travel lane. Hence the detection unit32changes the state to the state511, i.e., outside the travel lane, and discards the candidate for the travel lane set at the position of interest. Conversely, when the state of the position of interest is the state514, i.e., possibly inside the travel lane, and the order of the types of objects in the pixel group of interest corresponds to case 2 (the object in the right pixel is the travel lane), the position of interest should be inside the travel lane. Hence the detection unit32changes the state to the state512, i.e., inside the travel lane. Additionally, the detection unit32formally determines that the candidate for the travel lane set at the position of interest is inside the travel lane, and updates the assumed position of the right boundary of the travel lane to the position of the right pixel of the pixel group of interest. When the state of the position of interest is the state512, i.e., inside the travel lane, and the order of the types of objects in the pixel group of interest corresponds to case 8 or 13 (the object in the right pixel is a lane line or a different road surface), the detection unit32sets the position of the left pixel of the pixel group of interest as a valid position of the right boundary of the travel lane, and changes the state to the state511, i.e., outside the travel lane. When the state of the position of interest is the state512, i.e., inside the travel lane, and the order of the types of objects in the pixel group of interest corresponds to case 10, 14, or 15, the detection unit32sets the position of the left pixel of the pixel group of interest as an invalid right boundary of the travel lane, and changes the state to the state511, i.e., outside the travel lane. Regarding other combinations of the state of the position of interest and the order of objects in the pixel group of interest, state transitions occur according to the state transition diagram500.

Thereafter, while shifting the position of interest to the right pixel by pixel, the detection unit32similarly sets the pixel group of interest whose left pixel is the position of interest, and changes the state of the position of interest according to the state transition diagram500, based on the current state of the position of interest and the order of the types of objects in the pixel group of interest. When the position of interest reaches the right end of the pixel line being scanned, the detection unit32detects the validly registered position of the left boundary of the travel lane, if any, as the position thereof in this pixel line. Similarly, the detection unit32detects the validly registered position of the right boundary of the travel lane, if any, as the position thereof in this pixel line. In the case that the position of the left boundary of the travel lane is not validly registered and is only invalidly registered when the position of interest reaches the right end of the pixel line being scanned, the detection unit32does not detect the left boundary of the travel lane in this pixel line. Similarly, in the case that the position of the right boundary of the travel lane is not validly registered and is only invalidly registered when the position of interest reaches the right end of the pixel line being scanned, the detection unit32does not detect the right boundary of the travel lane in this pixel line.

In this way, the detection unit32can detect the left and right boundaries of the travel lane by only one scan per pixel line.

When a lane line and a different marking parallel thereto, such as a guiding lane marking, are provided, they do not adjoin, and thus in the image also, pixels representing the travel lane should exist between pixels representing the lane line and pixels representing the different marking. However, an object farther from the vehicle10or the camera2is represented smaller in the image. Thus, in the region in the image corresponding to the position on the road surface a certain distance or more away from the vehicle10, it may be difficult to identify the portion of the road surface of the travel lane between the lane line and the different marking, and a pixel representing the lane line may adjoin a pixel representing the different marking. Thus, for a pixel line in the image corresponding to a position a predetermined distance or more away from the vehicle10, the detection unit32may detect the position of the different marking as a boundary of the travel lane when the identified types of objects in respective pixels are arranged in the order of the lane line, the different marking, and the travel lane from the side away from the vehicle10, as in case 1 or 13 illustrated inFIG.5. For example, the predetermined distance may be such that the resolution of images obtained by the camera2will hinder identification of the road surface of the travel lane between a lane line and a different marking in the images.

FIG.6illustrates an example of an image representing a lane line and a different marking. As illustrated inFIG.6, the image600represents a guiding lane marking602along a lane line601with a certain interval in between. However, in the result610of identification of the types of objects in respective pixels in the image600, the pixel group representing the lane line601adjoins the pixel group representing the guiding lane marking602at the position corresponding to a region620because it is too far from the vehicle10. Even in the region620, the detection unit32can correctly detect a boundary of the travel lane by detecting, as described above, the position of the different marking as that of a boundary of the travel lane when the identified types of objects in respective pixels are arranged in the order of the lane line, the different marking, and the travel lane, i.e., when the lane line adjoins the different marking. For this reason, the detection unit32can detect a boundary of the travel lane from the vehicle10to a farther position. Additionally, at a position not very far from the vehicle10, the detection unit32determines the position of the travel lane in a pixel line where the lane line or the different road surface and then the travel lane are arranged in this order from the side away from the vehicle as a boundary of the travel lane, as described above. Thus, even if a different marking, such as a guiding lane marking, is provided along a lane line, the detection unit32can prevent the boundary between the different marking and the travel lane from being erroneously detected as the real boundary of the travel lane.

A lane line or a different marking, such as a guiding lane marking, may be blurred, depending on the state of the surface of the road being traveled by the vehicle10. In such a case, the accuracy of identification of the types of objects in respective pixels by the classifier may be low.

FIG.7illustrates an example of an image of a road with a blurred lane line and the result of identification of the types of objects in respective pixels. In the road represented in the image700, a lane line701and a different marking702are blurred. For this reason, the result710of identification of the types of objects in respective pixels in the image700includes pixels representing the different marking702but erroneously identified as those representing the lane line, and pixels representing a different road surface but erroneously identified as those representing the travel lane. In such a case, the position of a boundary line of the travel lane may also be erroneously detected. In particular, if pixels included in a region adjacent to the travel lane with a lane line in between are erroneously identified as those representing the travel lane, multiple regions determined as the inside of the travel lane (hereafter, “lane regions”) may be detected in a pixel line.

Thus, in a pixel line from which multiple lane regions are detected as a result of a scan, the detection unit32selects the most reliable one of the lane regions. Of the left and right endpoints of the selected lane region, the detection unit32detects one on whose side no adjacent lane region exists between an edge of the image and the selected lane region as a valid boundary of the travel lane, and detects the other on whose side an adjacent lane region exists between the opposite edge of the image and the selected lane region as an invalid boundary of the travel lane. For example, the detection unit32determines a wider lane region as a more reliable one. For example, in a pixel line720illustrated inFIG.7, two lane regions721and722are detected; of these, the right lane region722is wider than the left lane region721. Hence the right lane region722is selected. Since no lane region exists on the right of the lane region722, the right endpoint of the lane region722is detected as a valid position of a boundary of the travel lane. In contrast, since the other lane region721exists on the left of the lane region722, the left endpoint of the lane region722is determined as an invalid boundary. In this way, the detection unit32can prevent erroneous detection of the position of a boundary of the travel lane even if a lane line or other markings represented in the image are blurred.

According to a modified example, in a pixel line from which multiple lane regions are detected in the latest image, the detection unit32may determine one of the lane regions such that the difference between this lane region and a lane region in the pixel line in the immediately preceding image closest to the former pixel line is the smallest, as the most reliable lane region. Alternatively, when multiple sets of contiguous pixels are inside the travel lane, the detection unit32may combine these sets of pixels into one and determine the left and right endpoints of the combined lane region as the positions of the boundaries of the travel lane.

The detection unit32notifies the vehicle control unit33of the detected valid positions of the boundaries of the travel lane for each pixel line.

The vehicle control unit33controls travel of the vehicle10, based on the positions of the boundaries of the travel lane detected for each pixel line. For example, the vehicle control unit33generates a planned trajectory of the vehicle10in a predetermined section from the current position of the vehicle10to a predetermined distance away, based on the positions of the left and right boundaries of the travel lane detected for each pixel line, so that the vehicle10will travel on the center of the travel lane. To this end, the vehicle control unit33determines an approximate line obtained by applying, for example, the least-squares method to the set of pixels representing the left boundary of the travel lane, as the left boundary line of the travel lane. Similarly, the vehicle control unit33determines an approximate line obtained by applying, for example, the least-squares method to the set of pixels representing the right boundary of the travel lane, as the right boundary line of the travel lane. The position of each pixel in an image obtained by the camera2corresponds to the direction to an object represented in the pixel viewed from the camera2. Thus the vehicle control unit33sets a planned trajectory so that the position in the image corresponding to the center line along the travel direction of the vehicle10will be equidistant from the left and right boundary lines of the travel lane. The vehicle control unit33then controls components of the vehicle10so that the vehicle10will travel along the planned trajectory. For example, the vehicle control unit33determines the steering angle for the vehicle10to travel along the planned trajectory, and outputs a control signal depending on the steering angle to an actuator (not illustrated) that controls the steering wheel of the vehicle10. Additionally, the vehicle control unit33determines the acceleration of the vehicle10, based on the current speed of the vehicle10measured by a vehicle speed sensor (not illustrated) or the acceleration of the vehicle10measured by an acceleration sensor (not illustrated), so that the vehicle10will travel while keeping a speed designated by the driver, a speed set according to the legally permitted speed of the road being traveled by the vehicle10, or the distance between the vehicle10and a leading vehicle constant. The vehicle control unit33sets the degree of accelerator opening or the amount of braking so that the acceleration of the vehicle10will be equal to the determined acceleration. The vehicle control unit33then determines the amount of fuel injection according to the set degree of accelerator opening, and outputs a control signal depending on the amount of fuel injection to a fuel injector of the engine of the vehicle10. Alternatively, the vehicle control unit33outputs a control signal depending on the set amount of braking to the brake of the vehicle10.

FIG.8is an operation flowchart of the vehicle control process including the boundary detection process and executed by the processor23. Whenever receiving an image from the camera2, the processor23executes the vehicle control process in accordance with the operation flowchart illustrated inFIG.8. In the operation flowchart described below, the process of steps S101and S102corresponds to the boundary detection process.

The identification unit31of the processor23inputs an image into a classifier to identify the types of objects represented in respective pixels of the image (step S101).

The detection unit32of the processor23detects the positions of the left and right boundaries of the travel lane by determining, for each of pixel lines in a direction crossing the travel lane in the image, whether the position corresponding to a pixel group of interest including a predetermined number of contiguous pixels is inside the travel lane in order along a scanning direction from one end to the other end of the pixel line, depending on the order of the types of objects represented in the pixel group of interest and the result of determination whether the position corresponding to the immediately preceding pixel group with respect to the scanning direction is inside the travel lane (step S102).

The vehicle control unit33of the processor23controls the vehicle10, based on the positions of the left and right boundaries of the travel lane detected in each pixel line, so that the vehicle10will travel along the travel lane (step S103). The processor23then terminates the vehicle control process.

As has been described above, the apparatus for detecting a lane boundary inputs an image into a classifier that has been trained to identify the type of object represented in each pixel, thereby identifying the types of objects represented in respective pixels. The apparatus then detects the positions of the left and right boundaries of the travel lane by determining, for each of pixel lines in a direction crossing the travel lane, whether the position corresponding to a pixel group of interest including a predetermined number of contiguous pixels is inside the travel lane in order along a scanning direction from one end to the other end of the pixel line, depending on the order of the types of objects represented in the pixel group of interest and the result of determination whether the position corresponding to the immediately preceding pixel group with respect to the scanning direction is inside the travel lane. In this way, the apparatus can detect the boundaries of the travel lane by one scan per pixel line, reducing the computational burden. Additionally, the apparatus only has to scan each pixel line along a particular direction, and thus can match the order of pixels of the image stored in the memory with the scanning direction, improving the efficiency of memory access.

According to a modified example, when the positions of a boundary of the travel lane are separated a predetermined distance or more between adjoining pixel lines in the same image, the detection unit32may determine that the position of the boundary of the travel lane in one of the pixel lines is invalid. For example, when two pixel lines between which the positions of a boundary of the travel lane are separated a predetermined distance or more are detected, the detection unit32may determine that the position of the boundary in the pixel line corresponding to the position farther from the vehicle10is invalid. Alternatively, when the positions of a boundary of the travel lane between pixel lines at the same position in two sequential images are separated a predetermined distance or more, the detection unit32may also determine that the position of the boundary of the travel lane in the pixel line of the newer image is invalid.

A computer program for achieving the functions of the units of the processor23of the apparatus according to the embodiment may be provided in a form recorded on a computer-readable and portable medium, such as a semiconductor memory, a magnetic recording medium, or an optical recording medium.

As described above, those skilled in the art may make various modifications according to embodiments within the scope of the present invention.