Detection and planar representation of three dimensional lanes in a road scene

A vehicle, system for operating a vehicle and method of navigating a vehicle. The system includes a sensor and a multi-layer convolutional neural network. The sensor generates an image indicative of a road scene of the vehicle. The multi-layer convolutional neural network generates a plurality of feature maps from the image via a first processing pathway, projects at least one of the plurality of feature maps onto a defined plane relative to a defined coordinate system of the road scene to obtain at least one projected feature map, applies a convolution to the at least one projected feature map in a second processing pathway to obtain a final feature map, and determines lane information from the final feature map. A control system adjusts operation of the vehicle using the lane information.

INTRODUCTION

The subject disclosure relates to detection of lanes in a road scene. Commonplace technologies rely or machine-based systems and techniques to detect a lane in road scenes. Such systems and techniques can utilize machine-learning frameworks to infer road elements separately in an image domain. Then, heuristics or other types of empirical modeling are applied to combine those road elements into a lane or a group of lanes in the road scene. Further, such systems and techniques typically adopt an independent-sensor approach in which a group of lanes is detected utilizing a defined type of sensing modality (camera sensing, LIDAR sensing, or the like). Various groups of lanes detected in respective sensing modalities are usually fused after lane detection has been completed for each sensing modality. Not only can the detection and/or representation of a lane depend greatly on the types of heuristics applied to a group of detected road elements, but fusing sensor data a posteriori can diminish detection and/or representation fidelity. Poor fidelity in lane detection and/or representation can complicate or impede automated or autonomous operation of a vehicle circulating on a road. Accordingly, it is desirable to provide technologies for detection of lanes in a road scene.

SUMMARY

In one exemplary embodiment, a method of navigating a vehicle is disclosed. An image is obtained, the image indicative of a road scene at a sensor of the vehicle. A plurality of feature maps are generated from the image via a first processing pathway of a multi-layer convolutional neural network. At least one of the feature maps is projected onto a defined plane relative to a defined coordinate system of the road scene to obtain at least one projected feature map. A convolutions is applied to the at least one projected feature map in a second processing pathway of the multi-layer convolutional neural network to obtain a final feature map. The lane information is determined from the final feature map.

In addition to one or more of the features described herein, the image can include input data having a plurality of sensor modalities. In an embodiment in which the image further comprises a plurality of images, the first processing pathway to each of the plurality of images to obtain the plurality of feature maps for each of the images, projecting the plurality of feature maps onto the defined plane, combining the projected feature maps, and applying the convolution to the combined projected feature maps. The multi-layer convolutional neural network includes an encoder-decoder network. The final feature map is horizontally invariant and determining lane information further comprises determining a three-dimensional representation of the lanes. Projecting the at least one of the feature maps includes applying a homographic transformation to the at least one of the feature maps. The lane information is supplied to a control system configured to adjust operation of the vehicle using the lane information.

In another exemplary embodiment, a system for operating a vehicle is disclosed. The system includes a sensor and a multi-layer convolutional neural network. The sensor is configured to generate an image indicative of a road scene of the vehicle. The multi-layer convolutional neural network is configured to generate a plurality of feature maps from the image via a first processing pathway, project at least one of the plurality of feature maps onto a defined plane relative to a defined coordinate system of the road scene to obtain at least one projected feature map, apply a convolution to the at least one projected feature map in a second processing pathway to obtain a final feature map, and determine lane information from the final feature map.

In addition to one or more of the features described herein, the image includes input data from sensors having different sensor modalities. In one embodiment, the image includes a plurality of images, and the neural network is further configured to apply the first processing pathway to each of the plurality of images to obtain the plurality of feature maps for each of the plurality of images, project the plurality of feature maps onto the defined plane, combine the projected feature maps, and apply the convolution to the combined projected feature maps. The multi-layer convolutional neural network includes an encoder-decoder network. The final feature map is horizontally invariant, the system further comprising a lane representation module configured to determine a three-dimensional representation of the lanes from the final feature map. The system further includes a projection module configured to project the at least one of the feature maps by applying a homographic transformation to the at least one of the feature maps. The system further includes a control system configured to adjust operation of the vehicle using the lane information.

In yet another exemplary embodiment, the disclosure provides a vehicle is disclosed. The vehicle includes a sensor, a multi-layer convolutional neural network and a control system. The sensor is configured to generate an image indicative of a road scene of the vehicle. The multi-layer convolutional neural network is configured to generate a plurality of feature maps from the image via a first processing pathway, project at least one of the plurality of feature maps onto a defined plane relative to a defined coordinate system of the road scene to obtain at least one projected feature map, apply a convolutions to the at least one projected feature map in a second processing pathway to obtain a final feature map, and determine lane information from the final feature map. The control system is configured to adjust operation of the vehicle using the lane information.

In addition to one or more of the features described herein, the image includes input data from sensors having different sensor modalities. In an embodiment in which the image includes a plurality of images, the neural network is further configured to apply the first processing pathway to each of the plurality of images to obtain the plurality of feature maps for each of the plurality of images, project the plurality of feature maps onto the defined plane, combine the projected feature maps, and apply the convolution to the combined projected feature maps. The multi-layer convolutional neural network includes an encoder-decoder network. The final feature map is horizontally invariant, the system further comprising a lane representation module configured to determine a three-dimensional representation of the lanes from the final feature map. The vehicle further includes a projection module configured to project the at least one of the plurality of feature maps by applying a homographic transformation to the at least one of the plurality of feature maps.

DETAILED DESCRIPTION

The disclosure recognizes and addresses, in at least some embodiments, the issue of detection of a lane in a road scene. Embodiments of this disclosure include systems, vehicles, and methods that, individually or in combination, permit or otherwise facilitate detection of a group of lanes in a road scene. More specifically, yet not exclusively, a dual-pathway neural network (DNN) can operate on first feature maps derived from at least a world-view plane of the road scene and second feature maps based at least on a defined plane (e.g., top-view projection) within a world-view/global coordinate system. The DNN can jointly process the first feature maps and the second feature maps to generate a planar representation of a group of lanes in the road scene. While some embodiments of the disclosure are illustrated with reference to a road scene, the disclosure is not so limited. Indeed, the principles and practical elements disclosed herein can be applied to other types of thoroughfare scenes, such as a street scene, a highway scene, or the like.

With reference to the drawings,FIG. 1depicts an operational environment100for detection of a lane within a road scene105, in accordance with an embodiment. The road scene105includes a volume defined by a global coordinate system G having defining vectors (û1, û2, û3), where û1is a vector crossing the line of sight of a vehicle152, û2is a vector extending along the direction of travel of the vehicle115and û3is a vector extending vertically.

The road scene105can include a road102having a group of lanes104(including, in some instance, one or more edge lanes). The road102also can include a sidewalk portion or a shoulder portion, either one of such portions being adjacent to the road. The road102also can include, for example, multiple objects located on the road and/or in a vicinity thereof. For instance, a first object can be a vehicle106(stationary or in motion) positioned relative to the vehicle115on the road102; a second object can be a stationary structure108, such as a tree, a utility post, a building, or the like; and a third object can include a human110, either a pedestrian or an operator of a vehicle (motorized or otherwise).

Vehicle115can detect a lane104on the road102. The vehicle115can detect the lane104while traversing the road102or while maneuvering (e.g., parking) on the road102. The vehicle115also can, in some instances, detect the lane104while being stationary. The vehicle115includes a sensing platform120that can generate imaging data representative of the environment of the vehicle115. The imaging data can be generated from signals detected by the sensing platform120. The signals can include one or more types of electromagnetic (EM) signals (e.g., visible light, infrared light, or radio waves). More specifically, the sensing platform120can include a sensor system130that can detect EM signals at a defined rate f (a real number in units of frequency). Thus, imaging data (analog or digital) generated in response to detected EM signals can be organized in frames. A frame is, or includes, a data structure that contains one or more datasets generated in response to signals detected at a defined instant or during a defined period. As such, a frame corresponds to a defined instant during a detection interval.

The sensor system130can include multiple sensor devices that provide (e.g., generate and/or output) sensor signals. The sensor devices can be arranged or otherwise configured about the vehicle115. In some embodiments, the multiple sensor devices can be homogenous and can generate an output sensor signal of a defined type. Thus, the sensor system130can generate data of a defined type. For example, each one of the multiple sensor devices can include a camera device that senses photons in the visible portion of the electromagnetic (EM) radiation spectrum, and the sensor system130can embody a camera system that generates imaging data representative or otherwise indicative of a region relative to the vehicle115. As another example, each one of the multiple sensor devices can include light source devices (e.g., infrared laser devices) and photodetector devices, and the sensor system130can embody a light detection and ranging (LIDAR) sensor system that generates other imaging data representative or otherwise indicative of the road scene105.

In other embodiments, the multiple sensor devices can be heterogeneous and can generate and output sensor signals of various types. For instance, the multiple sensor devices can include a first type of sensor devices and a second type of sensor devices. Thus, in one aspect, the sensor system130is constituted by sensor systems having respective types of sensor devices. Each sensor system embodies or constitutes a sensing system of a defined type—e.g., a camera system, a radar system, a LIDAR system, a sonar system, a thermal mapping system, or the like—that operates in a defined imaging modality. Accordingly, each sensor system can provide a defined type of imaging data representative of the road scene105. Regardless of the specific sensing architecture/modality, the sensor system130can provide (e.g., send and/or make available) imaging data135indicative or otherwise representative of the road scene105at defined frames.

As is illustrated inFIG. 1, the sensing platform120includes a lane detection system140that can use at least a portion of the imaging data135to identify a lane104within the road scene105, on a defined plane Π within the global coordinate system G. In some embodiments, the defined plane Π corresponds to a top-view projection of the road scene105. In other words, the defined plane Π is a plane defined by vectors û1and û2(i.e., orthogonal to vector û3). More specifically, the lane detection system140includes a dual-pathway neural network (DNN) module144and a projection module148for transformation images between vantage points. The DNN module144and projection module148can output lane information representative of a group of lanes identified on the defined plane Π. This output can be used to process imaging data135obtained by sensors system130to obtain planar lane representations150that can be used to determine lane information160. The lane information160can be retained as one or more data structures containing the planar lane representations150.

The vehicle115includes a control system117communicatively coupled to the lane detection system140. The control system117can receive or access data from the lane detection system140such as lane information160and can augment, automate or otherwise control the vehicle115to navigate the road102based on the lane information160, for example, to remain within a lane104, etc.

FIG. 2illustrates an operational architecture of the DNN200and projection module148of the lane detection system140. The DNN200includes a first processing pathway230and a second processing pathway240. The first pathway230is used to determine feature maps in a first domain (e.g., the world-view domain) based on images obtained from sensors of the sensor system130having the viewpoint within the world-view coordinate system. The second pathway NN module240operates on feature maps within a second domain or top-view domain as seen from a top-view vantage point or bird's eye view vantage point of the road scene105,FIG. 1, such as plane Π. A feature map can be a three-dimensional tensor indicating channel and x, y coordinates in an image. A homographic transformation is used to obtain features maps in the top-view domain from feature maps obtained in the world-view domain.

The first processing pathway230includes a convolutional neural network (CNN) that generates respective feature maps during the first-pathway. First processing pathway230shows an illustrative image202obtained from the sensor system130,FIG. 1and a first feature map204obtained from the image202. The first feature map is provided to the CNN of the first processing pathway230, which applies a convolution to the first feature map204to obtain a second feature map206. Continuing this process, convolution of the second feature map206generates a third feature map208and convolution of the third feature map208generates a fourth feature map210.

The projection module148,FIG. 1transforms a first point within a plane corresponding to a world view of the road scene105into a second point within the defined plane Π. More concretely, the projection module148transforms a point p in a feature map generated by the first processing pathway230into a point p′ within the defined plane Π. The projection module148can transform p into p′ by applying a defined homographic transformation that maps the world view of the road scene105onto the defined plane Π. Therefore, the projection module148can receive a feature map from the first processing pathway230and can project the feature map onto the defined plane Π, thus generating a projected feature map in the second processing pathway240.

The homographic transformation is applied to each of the feature maps204,206,208,210in the world-view domain to generate projected feature maps in the bird's-eye view domain. (The homographic transformations are indicated by vertical arrows extending from features maps204,206,208, and210). Homographic transformation of first feature map204generates first projected feature map204p, homographic transformation of second feature map206generates second projected feature map206p, homographic transformation of third feature map208generates third projected feature map208pand homographic transformation of fourth feature map210generates further projected feature map210p. While four feature maps are show inFIG. 2for illustrative purposes, any number of feature maps can be created using the CNN along the first pathway230.

In the second processing pathway240, convolution of the first projected feature map204pgenerates a second top-view feature map216within the top-view domain240. The second top-view feature map216represents a convolution of a homographic transformation of the first feature map204. Meanwhile, the second projected feature map206prepresents a homographic transformation of a convolution of the first feature map204. The second top-view feature map216is combined or concatenated with the second projected feature map206pin order to obtain a combined feature map206c. A convolution is then performed on the combined feature map206cin order to generate a third top-view feature map218. The third top-view feature map218is combined or concatenated with the third projected feature map208to form a combined feature map208cand a convolution is performed on the combined feature map208cto obtain a fourth top-view feature map220.

The fourth top-view feature map220is combined or concatenated with the fourth projected feature map210pto form combined feature map210c. In the top-view domain, the convolutions can continue from the combined feature map210cin the second processing pathway240without further concatenation steps, extending the number of feature maps in the top-view domain beyond that of the world-view domain. These additional convolutions reduce the y-dimension of the three-dimensional tensor thereby creating a two-dimensional tensor defined by an indicating channel and an x-coordinate. In the illustrative DNN ofFIG. 2, subsequent convolutions in the second processing pathway240produces fifth top-view feature map222, sixth top-view feature map224and seventh top-view feature map226. A lane within the road scene105can be determined from the top-view feature map226.

FIG. 3schematically depicts a system300for a lane detection system140,FIG. 1for detection and planar representation of a lane within road scenes using imaging data from sensors having different sensing modalities. The data from each sensor is fused into a common two-dimensional (2D) coordinate system (ê1, ê2). The 2D coordinate system is orthogonal and permits representing a position vector within a top-view plane (e.g., the defined plane Π).

The system300includes multiple sensor systems generally having different sensing modalities, including a sensor system3101, a sensor system3102, . . . , and a sensor system310N. Each one of the multiple sensor systems can provide (e.g., send and/or make available) a defined type of input data or image representative of a road scene (e.g., road scene105inFIG. 1) at a defined frame. In one embodiment, the sensor system3101provides an image3151; the sensor system3102provides an image3152, . . . , and the sensor system310Nprovides an image315N. In one embodiment, the sensor system3101, sensor system3102, and sensor system310Nembody or constitute a camera system, a LIDAR system, and a radar system, respectively. In some embodiments, the sensing platform120,FIG. 1can have only two sensor systems, e.g., sensor system3101(e.g., a camera system) and sensor system3102(e.g., a LIDAR system).

The lane detection system140can receive images3151-315Nfrom the sensors3101-310N. The DNN of the lane detection system140includes separate first processing pathways dedicated to each image, and a single second processing pathway that operates on the fusion of the data from the first processing pathways. In the illustrated system300, the lane detection system140includes first-pathway NN module3201, first-pathway NN module3202, . . . , and first-pathway NN module320N. Each of the first-pathway NN modules3201-320Nreceives respective input data or images3151-315Nand operates on the received images3151-315N. Operating on a received image can permit or otherwise facilitate, in some embodiments, semantic segmentation of a road scene (e.g., road scene105). As such, each one of the first-pathway NN modules3201-320Ncan include, for example, a CNN having multiple layers that generate respective feature maps3251-325Nin response to operating on a received image. Accordingly, each one of the first-pathway NN modules3201-320Ncan generate respective feature maps3251-325N. Each feature map3251-325Nresults from operation at a layer of the CNN corresponding to the respective first-pathway NN module3201-320N. As the sensor systems3101, . . . ,310Ncan include various input devices (e.g., camera system, LIDAR system, a radar system, etc.), the first-pathway NN modules3201, . . . ,320Nassociated with these sensor systems3101, . . . ,310Ncan include first-pathway NN modules suited for semantic segmentation of the particular input data or image (e.g., camera system, LIDAR system, a radar system, etc.).

To implement the second pathway NN module340, the projection module330receives feature maps3251-325N, and transforms each feature map3251-325Nin the group into a projected feature map onto a defined plane in the 2D coordinate system (ê1, ê2) using the homographic transformation discussed herein. The projection module148fuses the projected feature maps into a consolidated projected feature map335relative to the defined plane and provides the consolidated projected feature map335to second-pathway NN module340. In some embodiments, the second-pathway NN module340includes a CNN configured (e.g., defined and/or trained) to identify a lane on a defined plane Π in the 2D coordinate system (ê1, ê2) based at least on the consolidated projected feature map335. Output of the second-pathway NN module340can include lane information345(e.g., data and/or metadata) representative of a group of lanes identified on the defined plane Π.

FIG. 4illustrates a result of implementing the lane information system140discussed with respect toFIG. 3. The images402,404and406from sensors having different sensing modalities are combined to determine a bird's-eye view410that combines information from all three of the images402,404and406.

FIG. 5schematically depicts a system500for determining lane information at a lane information system140,FIG. 1using an encoder-decoder network. The encoder-decoder system500receives image502representative of a road scene in a world-view coordinate system. The encoder-decoder system500includes a convolutional encoder-decoder having an encoder network532and a decoder network534. A homographic transformation module550transforms at least one feature map formed in the encoder network532to form a corresponding bird's eye view feature map in the decoder network534by projecting the feature maps onto the defined plane Π.

The encoder network532generates a sequence of feature maps based on the image502. The encoder network532generates a feature map504afrom the image502. Additional feature maps are generated by applying a convolution followed by a batch renormalization and application of a rectified linear-non-linearity (ReLU). In particular, convolution, batch renormalization and ReLU are applied to feature map504ato obtain feature map504b. The encoder network532shows feature maps506obtained from sub-sampling of feature maps504, feature maps508obtained from sub-sampling of feature maps506, feature maps510obtained from sub-sampling of feature maps508and feature maps512obtained from sub-sampling of feature maps510.

At each stage max-pooling indices are captured and stored. Max-pooling is performed to achieve translation invariance over small spatial shifts in the input image. Max-pooling captures and stores boundary information in the encoder features map prior to sub-sampling the feature map. Max-pooling indices indicate the locations of the maximum feature values in a window of an encoder feature map.

The decoder network534regenerates the images in order to determine a feature map522csuitable for semantic segmentation. Feature maps516are regenerated from feature maps514, feature maps518are regenerated from feature maps516, feature maps520are regenerated from feature maps518and feature maps522a,522band522care regenerated from feature maps520. The pooling indices obtained at each stage in the encoder network532are used at the comparable stage of the decoder network534in order to obtain feature maps522a,522band522cthat can be provided to a classification layer524for semantic segmentation. As an example, the pooling indices504cfrom the initial stage of the encoder network532is provided to the final stage of the decoder network534.

The classification layer524determines confidence score (c; a real number) indicative of whether a lane is present the final feature maps522a,522band522c. The system500also can include a lane representation module536that determines a respective group of reference points along a direction orthogonal to the direction in which the defined portions are oriented. The lane representation module536outputs a bird's eye view image538of a region as projected onto the defined plane Π; the bird's eye view image538including lane markings or lane delimiters. Operation of the lane representation module536is described with respect toFIG. 6.

FIG. 6illustrates the bird's eye view image538of lane markings as indicated by the defined plane Π. The lane representation module536ofFIG. 5determines or locates the lane markings using the bird's eye view image538. The bird's eye view image538includes a first axis e1along the horizontal direction and a second axis e2along the vertical direction. The lane representation module536partitions the first axis e1into a set of N anchors {X1, X2, . . . , XN}, with the anchor along the first axis e1and corresponding to a column extending along the second axis e2. A set of K locations {y1, y2, . . . , yK} are defined along the second axis e2. Within each column {X1, X2, . . . , XN}, the lane representation module536determines whether there are any lane markings (e.g., delimiters602, centerlines604) within the column corresponding to the locations {y1, y2, . . . , yK}. The lane representation module536determines horizontal and elevation for lane markings {(x1,z1), (x2,z2), . . . , (xK,zK)} that correspond to the locations {y1, y2, . . . , yK} to define a set of three-dimensional points through which the lane marking passes. The values of {x1, x2, . . . , xK} are horizontal offsets relative to the anchor position. Therefore, the coordinates of the three-dimensional points can be written as {(Xn+x1, y1, z1), (Xn+x2, y2, z2), . . . , (Xn+xK, yK, zK)}. In addition, for each anchor or corresponding column, a confidence score c for the markings is determined. The lane representation module536compares the confidence score c to a defined threshold value cth. For bins in which the confidence score c is greater than or equal to the defined threshold value cth, the lane representation module536accepts the markings as being within the column and determines a parametric polynomial representation of the lane within the defined plane Π using at least the group of reference points within the bin. Although only three locations and corresponding road coordinates are shown inFIG. 6, any number of locations and road coordinates can be selected in alternative embodiments. Using three or more locations and road coordinates allows fitting polynomials through the road coordinates in order to determine lanes in three-dimensions. The process of determining the lane markings is horizontally invariant.

FIG. 7Apresents a top-view representation700illustrating multiple lanes that can be determined by the lane representation module536ofFIG. 5. The lane detection system140projects the group of lanes inFIG. 7Aonto a world view of the road scene including the detected group of lanes.FIG. 7Billustrates the results of such a projection, overlapping ground-truth data for the lanes in the road scene (open circles702) and projected detected lanes (thick lines704).

FIG. 8illustrates a result of three-dimensional road lane determination using the methods disclosed herein.FIG. 8Ashows an image802of a road scene that is used to produce three-dimensional lane representations.FIG. 8Bshows a three-dimensional graph804of the image802ofFIG. 8A. The ground-truth lane806markings are shown in the three-dimensional graph804. Also shown in the three-dimensional graph804are the three-dimensional lane representations808obtained using the methods disclosed herein.

FIG. 9presents a block diagram of an example of a computing system910to detect and represent a lane within a road scene, in accordance with aspects of this disclosure. The computing system910can include one or more processors920and one or more memory devices940(generically referred to as memory940) that include machine-accessible instructions (e.g., computer-readable and/or computer-executable instructions) that can be accessed and executed by at least one of the processor(s)920. In one example, the processor(s)920can be embodied in or can constitute a graphics processing unit (GPU), a plurality of GPUs, a central processing unit (CPU), a plurality of CPUs, an application-specific integrated circuit (ASIC), a microcontroller, a programmable logic controller (PLC), a field programmable gate array (FPGA), a combination thereof, or the like. In some embodiments, the processor(s)920can be arranged in a single computing apparatus (e.g., an electronic control unit (ECU), and in-car infotainment (ICI) system, or the like). In other embodiments, the processor(s)920can be distributed across two or more computing apparatuses (e.g., multiple ECUs; a combination of an ICI system and one or several ECUs; or the like).

The processor(s)920can be functionally coupled to the memory940by means of a communication structure930. The communication structure930is suitable for the particular arrangement (localized or distributed) of the processor(s)920. In some embodiments, the communication structure930can include one or more of bus architectures, such an Ethernet-based industrial bus, a controller area network (CAN) bus, a Modbus, other types of fieldbus architectures, or the like.

The memory940includes the lane detection system140. As such, machine-accessible instructions (e.g., computer-readable and/or computer-executable instructions) embody or otherwise constitute the lane detection system140. The machine-accessible instructions are encoded in the memory940and can be arranged in components that can be built (e.g., linked and compiled) and retained in computer-executable form in the memory940(as is shown) or in one or more other machine-accessible non-transitory storage media. At least one of the processor(s)920can execute the lane detection system140to cause the computing system910to detect and/or represent a group of lanes within a road scene in accordance with aspects of this disclosure.

Similarly, the memory940also can retain or otherwise store the control system950. As such, machine-accessible instructions (e.g., computer-readable and/or computer-executable instructions) embody or otherwise constitute the control system950. Again, the machine-accessible instructions are encoded in the memory940and can be arranged in components that can be built (e.g., linked and compiled) and retained in computer-executable form in the memory940(as is shown) or in one or more other machine-accessible non-transitory storage media. At least one of the one or more processors920can execute the control system950to cause the computing system910to implement a control process to adjust or otherwise control the operation of the vehicle115,FIG. 1, for example, or other types of vehicles. To that end, in one aspect, the control process can utilize or otherwise rely on a representation of one or more lanes generated by the lane detection system140.

While not illustrated inFIG. 9, the computing system910also can include other types of computing resources (e.g., interface(s) (such as I/O interfaces; controller devices(s); power supplies; and the like) that can permit or otherwise facilitate the execution of the software components. To that point, for instance, the memory940also can include programming interface(s) (such as application programming interfaces (APIs)), an operating system, firmware, and the like.

FIG. 10shows a flowchart illustrating a method1000for detecting a lane in a road scene in an embodiment of the invention. At block1002, an image indicative of a road scene of the vehicle is obtained at a sensor of the vehicle. The image can be a single image or a plurality of images. The plurality of images can be obtained from sensors having different sensor modalities. At block1004, a multi-layer convolution neural network is applied to the image, wherein a first processing pathway of the neural network generates a plurality of feature maps from the image. At block1006, the plurality of feature maps are projected onto a defined plane relative to a defined coordinate system of the road scene to obtain projected feature maps. At block1008, a second processing pathway of the neural network applies convolutions to the projected feature maps to obtain a final feature map. At block1010, lane information is determined from the final feature map.