Patent Description:
Lane detection technology is widely used in the fields of automatic driving, traffic surveillance, etc. In recent years, with the rapid development of deep learning, some machine learning models have been used in lane detection. For many existing lane detection methods, errors often occur in the detection. For example, objects such as railings, trees, vehicles, traffic identifiers and so on, are sometimes mistakenly identified as lanes. Many existing lane detection methods are only applied to a specific scenario but not in a varying environment (e.g., under strong light, weak light, under extreme weather). The robustness of many lane detection methods is highly questionable. Moreover, some objects, such as railings, sometimes have colors and shapes that are very similar to lane lines. These objects are often identified according to the surroundings, and thus many lane detection methods cannot be applied to the detection of lane lines with varying shapes (e.g., straight, curved, or folding). Therefore, it is desirable to provide systems and methods for detecting one or more lane lines of various shape(s) more accurately.

The document <CIT> (<NUM>-<NUM>-<NUM>) discloses a lane line detection method, including: acquiring an image to be detected; inputting the image to be detected into a neural network model to obtain a lane line prediction image, wherein the pixel value of each pixel in the predicted lane line image is used to indicate the lane line category to which the corresponding pixel in the image to be detected belongs; and the lane line information of the predicted lane line related to the image to be detected is determined based on the lane line prediction image. The document also discloses performing density clustering on pixels and fitting a lane line equation to pixels corresponding to the same predicted lane line in the cluster image.

According to an aspect of the present disclosure, a method for lane detection according to claim <NUM> is provided. The method is implemented on a computing device including at least one processor and at least one storage medium. The method includes obtaining an image, determining, for each of a plurality of pixels in the image, a semantic category using a trained semantic segmentation network. The method further includes determining one or more pixel sets based on the plurality of pixels according to a predetermined rule. Each of the one or more pixel sets includes one or more pixels of a same semantic category. The method further includes performing a binarization operation to obtain binary information of each of the plurality of pixels. For each of the one or more pixel sets, the method further includes performing a fitting operation on the one or more pixels in the pixel set to obtain a fitting line corresponding to a lane line; determining a position of the lane line based on the fitting line; and determining, based at least on the fitting line and the binary information of the one or more pixels in the pixel set a lane line category of the lane line.

In some embodiments, the performing a binarization operation to obtain binary information of each of the plurality of pixels may include performing an edge extraction operation on the plurality of pixels in the image to obtain edge information associated with each of the plurality of pixels, and determining, based on the edge information, the binary information of each of the plurality of pixels.

Determining, determining, based on the fitting line and the binary information of the one or more pixels in the pixel set, a lane line category of the lane line includes determining, for each of the one or more pixels in the pixel set, a searching range for the pixel; determining a reference number count of reference pixels, wherein there are no edge pixels near each of the reference pixels within the searching range; and determining the lane line category of the lane line by comparing the reference number count of pixels and a count threshold.

In some embodiments, the determining the lane line category of the lane line by comparing the reference number count of pixels and a count threshold may include in response to a determination that the reference number count of reference pixels is greater than the count threshold, determining that the lane line category of the lane line is a dashed line; and in response to a determination that the reference number count of pixels is less than or equal to the count threshold, determining that the lane line category of the lane line is a solid line.

Determining a searching range for the pixel includes determining a fitted coordinate of the pixel, and determining a searching range based on the fitted coordinate and a lane line width.

In some embodiments, the method may further include determining whether there are at least two pixel sets of the same semantic category in the one or more pixel sets; and in response to a determination that there are at least two pixel sets of the same semantic category in the one or more pixel sets, determining a difference between fitting parameters of the at least two pixel sets. The method may further include comparing the difference with a parameter difference threshold, and in response to a determination that the difference is less than or equal to the parameter difference threshold, generating a combined pixel set based on the at least two pixel sets.

In some embodiments, the fitting operation may be an iterative fitting operation. The method may further include, for each of the one or more pixel sets, determining whether there are one or more erroneous pixels that satisfy a predetermined condition. In response to a determination that there are one or more erroneous pixels, removing the one or more erroneous pixels from the pixel set to obtain the fitting line.

In some embodiments, the trained semantic segmentation network may include one or more convolutional layers and one or more deconvolutional layers. The one or more convolutional layers may be configured for a depth-wise separable convolution.

According to yet another aspect of the present disclosure, a system for lane detection according to claim <NUM> is provided. The system includes at least one storage device storing a set of instructions and at least one processor in communication with the storage device. When executing the set of instructions, the at least one processor is directed to cause the system to obtain an image and determine, for each of a plurality of pixels in the image, a semantic category using a trained semantic segmentation network. The at least one processor is further directed to cause the system to determine one or more pixel sets based on the plurality of pixels according to a predetermined rule and perform a binarization operation to obtain binary information of each of the plurality of pixels. Each of the one or more pixel sets includes one or more pixels of a same semantic category. The at least one processor is further directed to cause the system to, for each of the one or more pixel sets, perform a fitting operation on the one or more pixels in the pixel set to obtain a fitting line corresponding to a lane line; determine a position of the lane line based on the fitting line; and determine, based at least on the fitting line and the binary information of the one or more pixels in the pixel set, a lane line category of the lane line.

According to still another aspect of the present disclosure, a non-transitory computer readable medium according to claim <NUM> is provided. The non-transitory computer readable medium includes a set of instructions for lane detection. When executed by at least one processor, the set of instructions directs the at least one processor to effectuate a method. The method includes obtaining an image, determining, for each of a plurality of pixels in the image, a semantic category using a trained semantic segmentation network. The method further includes determining one or more pixel sets based on the plurality of pixels according to a predetermined rule. Each of the one or more pixel sets includes one or more pixels of a same semantic category. The method further includes performing a binarization operation to obtain binary information of each of the plurality of pixels. For each of the one or more pixel sets, the method includes performing a fitting operation on the one or more pixels in the pixel set to obtain a fitting line corresponding to a lane line; determining a position of the lane line based on the fitting line; and determining, based at least on the fitting line and the binary information of the one or more pixels in the pixel set, a lane line category of the lane line.

In the following detailed description, numerous specific details are set forth by way of examples in order to provide a thorough understanding of the relevant disclosure. However, it should be apparent to those skilled in the art that the present disclosure may be practiced without such details. In other instances, well-known methods, procedures, systems, components, and/or circuitry have been described at a relatively high-level, without detail, in order to avoid unnecessarily obscuring aspects of the present disclosure. Thus, the present disclosure is not limited to the embodiments shown, but to be accorded the widest scope consistent with the claims.

It will be understood that the term "system," "engine," "unit," "module," and/or "block" used herein are one method to distinguish different components, elements, parts, section or assembly of different level in ascending order. However, the terms may be displaced by other expressions if they may achieve the same purpose.

Generally, the word "module," "unit," or "block," as used herein, refers to logic embodied in hardware or firmware, or to a collection of software instructions. A module, a unit, or a block described herein may be implemented as software and/or hardware and may be stored in any type of non-transitory computer-readable medium or other storage devices. In some embodiments, a software module/unit/block may be compiled and linked into an executable program. It will be appreciated that software modules can be callable from other modules/units/blocks or from themselves, and/or may be invoked in response to detected events or interrupts. Software modules/units/blocks configured for execution on computing devices (e.g., processor <NUM> as illustrated in <FIG>) may be provided on a computer readable medium, such as a compact disc, a digital video disc, a flash drive, a magnetic disc, or any other tangible medium, or as a digital download (and can be originally stored in a compressed or installable format that needs installation, decompression, or decryption prior to execution). Such software code may be stored, partially or fully, on a storage device of the executing computing device, for execution by the computing device. Software instructions may be embedded in firmware, such as an EPROM. It will be further appreciated that hardware modules (or units or blocks) may be included in connected logic components, such as gates and flip-flops, and/or can be included in programmable units, such as programmable gate arrays or processors. The modules (or units or blocks) or computing device functionality described herein may be implemented as software modules (or units or blocks) but may be represented in hardware or firmware. In general, the modules (or units or blocks) described herein refer to logical modules (or units or blocks) that may be combined with other modules (or units or blocks) or divided into sub-modules (or sub-units or sub-blocks) despite their physical organization or storage.

It will be understood that when a unit, engine, module, or block is referred to as being "on," "connected to," or "coupled to" another unit, engine, module, or block, it may be directly on, connected or coupled to, or communicate with the other unit, engine, module, or block, or an intervening unit, engine, module, or block may be present, unless the context clearly indicates otherwise.

The terminology used herein is for the purposes of describing particular examples and embodiments only and is not intended to be limiting. It will be further understood that the terms "include" and/or "comprise," when used in this disclosure, specify the presence of integers, devices, behaviors, stated features, steps, elements, operations, and/or components, but do not exclude the presence or addition of one or more other integers, devices, behaviors, features, steps, elements, operations, components, and/or groups thereof.

In addition, it should be understood that in the description of the present disclosure, the terms "first", "second", or the like, are only used to distinguish the purpose of description, and cannot be interpreted as indicating or implying relative importance, nor can be understood as indicating or implying the order.

An aspect of the present disclosure relates to systems and/or methods for lane detection. As used herein, the term "lane detection" refers to the detection of one or more lanes on a road. The one or more lanes may be detected by identifying one or more lane lines corresponding to the one or more lanes. Thus, a system or method for lane detection may also be referred to as a system or method for lane line detection. An image associated with a scene on the road (also referred to as a "road image) may be obtained. To determine a lane, one or more lane lines may be identified from the road image. A trained semantic segmentation network may be used to determine a semantic category of each of a plurality of pixels in the road image. For example, the semantic segmentation network may be constructed based on a Mibilenet and a fully connected network (FCN). A portion of convolutional layers and a portion of the deconvolutional layers of the semantic segmentation network may be connected via skip connections, and thus more information of the road image (e.g., the edge) may be retained. Then one or more pixel sets may be determined based on the plurality of pixels according to a predetermined rule. Each of the one or more pixel sets may correspond to the semantic category. For each of the one or more pixel sets, the processing engine <NUM> may perform a fitting operation to obtain a fitting line corresponding to the lane line. In some embodiments, the fitting operation may be an iterative fitting operation that includes a plurality of iterations. In each iteration, it may be determined whether there are one or more erroneous pixels. In response to a determination that there are one or more erroneous pixels, the one or more erroneous pixels may be removed. The removal of the erroneous pixels may increase the accuracy of the fitting line corresponding to a lane line, and thus the position of the lane line may be more accurately determined.

In some cases, the lane line category (e.g., a dashed line, a solid line, a straight line, or a curved line) of the lane line may be determined by the trained semantic segmentation network. In some cases, a binarization operation may be performed on the road image to obtain binary information based on edge information of each of the plurality of pixels in the road image. The lane line category may be determined based on the binary information of each of the plurality of pixels in the road image. In some cases, an error in the lane line category determined by the trained semantic segmentation network may be corrected. For example, if two pixel sets of the same semantic category have similar fitting parameters, the two pixel sets may correspond to the same lane line, and thus the two pixel sets may be combined to obtain a combined pixel set. The semantic category of the combined pixel set may be determined based on the binary information of each pixel in the combined pixel set. Therefore, the accuracy of determining the lane lines may be improved.

<FIG> is a schematic diagram illustrating an exemplary system for image processing according to some embodiments of the present disclosure. As shown, the system <NUM> may include a server <NUM>, a storage device <NUM>, an acquisition device <NUM>, a user terminal <NUM>, and a network <NUM>.

The server <NUM> may process information and/or data relating to the system <NUM> to perform one or more functions described in the present disclosure. In some embodiments, the server <NUM> may include one or more processing engines <NUM> (e.g., single-core processing engine(s) or multi-core processor(s)). In some embodiments, the processing engine <NUM> may be configured to detect one or more lanes in a road image by determining one or more lane lines in the road image. In some embodiments, the processing engine <NUM> may determine the semantic category of each of a plurality of pixels in the road image. The processing engine <NUM> may further determine one or more pixel sets based on the plurality of pixels according to a predetermined rule. For each of the one or more pixel sets, the processing engine <NUM> may perform a fitting operation to obtain a fitting line corresponding to the lane line. Merely by way of example, the processing engine <NUM> may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), an application-specific instruction-set processor (ASIP), a graphics processing unit (GPU), a physics processing unit (PPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a controller, a microcontroller unit, a reduced instruction-set computer (RISC), a microprocessor, or the like, or any combination thereof.

The server <NUM> may be a single server or a server group. The server group may be centralized, or distributed (e.g., server <NUM> may be a distributed system). In some embodiments, the server <NUM> may be local or remote. For example, the server <NUM> may access information and/or data stored in the acquisition device <NUM>, and/or the storage device <NUM> via the network <NUM>. As another example, the server <NUM> may be directly connected to the acquisition device <NUM>, and/or the storage device <NUM> to access stored information and/or data. In some embodiments, the server <NUM> may be implemented on a cloud platform. Merely by way of example, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, a multi-cloud, or the like, or any combination thereof. In some embodiments, the server <NUM> may be implemented on a computing device <NUM> having one or more components illustrated in <FIG> of the present disclosure.

The storage device <NUM> may store data and/or instructions. The data and/or instructions may be obtained from, for example, the server <NUM>, the acquisition device <NUM>, and/or any other component of the system <NUM>.

In some embodiments, the storage device <NUM> may store data and/or instructions that the server <NUM> may execute or use to perform exemplary methods described in the present disclosure. For example, the storage device <NUM> may store an image to be processed. As another example, the storage device <NUM> may store a trained semantic segmentation network. As another example, the storage device <NUM> may store an adjusted image generated after processing. In some embodiments, the storage device <NUM> may include a mass storage, a removable storage, a volatile read-and-write memory, a read-only memory (ROM), or the like, or any combination thereof. Exemplary mass storage may include a magnetic disk, an optical disk, solid-state drives, etc. Exemplary removable storage may include a flash drive, a floppy disk, an optical disk, a memory card, a zip disk, a magnetic tape, etc. Exemplary volatile read-and-write memory may include a random access memory (RAM). Exemplary RAM may include a dynamic RAM (DRAM), a double date rate synchronous dynamic RAM (DDR SDRAM), a static RAM (SRAM), a thyristor RAM (T-RAM), and a zero-capacitor RAM (Z-RAM), etc. Exemplary ROM may include a mask ROM (MROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a compact disk ROM (CD-ROM), and a digital versatile disk ROM, etc. In some embodiments, the storage device <NUM> may be implemented on a cloud platform. Merely by way of example, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, a multi-cloud, or the like, or any combination thereof.

In some embodiments, the storage device <NUM> may be connected to the network <NUM> to communicate with one or more components of the system <NUM> (e.g., the server <NUM>, the acquisition device <NUM>). One or more components of the system <NUM> may access the data or instructions stored in the storage device <NUM> via the network <NUM>. In some embodiments, the storage device <NUM> may be directly connected to or communicate with one or more components of the system <NUM> (e.g., the server <NUM>, the acquisition device <NUM>). In some embodiments, the storage device <NUM> may be part of another component of the system <NUM>, such as the server <NUM>, the acquisition device <NUM>, or the user terminal <NUM>.

In some embodiments, one or more components of the system <NUM> (e.g., the server <NUM>, the acquisition device <NUM>) may have permission to access the storage device <NUM>. For example, the server <NUM> or the user terminal <NUM> may obtain the image to be processed from the storage device <NUM>.

The acquisition device <NUM> may be and/or include any suitable device that is capable of acquiring an image. In some embodiments, the acquisition device <NUM> may include a mobile phone <NUM>-<NUM>, a computer <NUM>-<NUM>, a surveillance camera <NUM>-<NUM>, etc. The computer <NUM>-<NUM> may include but not limited to a laptop, a tablet computer, a desktop, or the like, or any combination thereof. The surveillance camera <NUM>-<NUM> may include but not limited to a gun camera, a dome camera, an integrated camera, a monocular camera, a binocular camera, a multi-view camera, or the like, or any combination thereof. The image acquired by the acquisition device <NUM> may be a single image or a frame of a video. In some embodiments, the acquisition device <NUM> may include a plurality of components each of which can acquire an image. For example, the acquisition device <NUM> may include a plurality of sub-cameras that can take pictures or videos simultaneously.

The user terminal <NUM> may be associated with a user. Exemplary terminal devices <NUM> may include a mobile phone <NUM>-<NUM>, a computer <NUM>-<NUM>, a tablet computer <NUM>-<NUM>, or the like. In some embodiments, the user terminal <NUM> may be and/or include any suitable device that can display or output information in a human-readable form, such as text, image, audio, video, graph, animation, or the like, or any combination thereof. In some embodiments, the user may view information and/or input data and/or instructions via the user terminal <NUM>. For example, the user may view the road image on a display device of the user terminal <NUM>. As another example, the user may input an instruction to start lane detection via the user terminal <NUM>. The display device of the user terminal <NUM> may include a cathode ray tube (CRT) display, a liquid crystal display (LCD), a light emitting diode (LED) display, a plasma display panel (PDP), a 3D display, or the like. In some embodiments, the user terminal <NUM> may be connected to one or more components of the system <NUM> (e.g., the server <NUM>, the storage device <NUM>, the acquisition device <NUM>) via the network <NUM>, such as a wireless network or a wired network (e.g., a coaxial cable network).

The network <NUM> may include any suitable network that can facilitate the exchange of information and/or data for the system <NUM>. In some embodiments, one or more components in the system <NUM> (e.g., the server <NUM>, the storage device <NUM>, and the acquisition device <NUM>) may send information and/or data to another component(s) in the system <NUM> via the network <NUM>. For example, the server <NUM> may obtain/acquire images from the acquisition device <NUM> via the network <NUM>. In some embodiments, the network <NUM> may be any type of wired or wireless network, or combination thereof. Merely by way of example, the network <NUM> may include a cable network (e.g., a coaxial cable network), a wireline network, an optical fiber network, a telecommunications network, an intranet, an Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a wide area network (WAN), a public telephone switched network (PSTN), a Bluetooth network, a ZigBee network, a near field communication (NFC) network, or the like, or any combination thereof.

It should be noted that the above description is merely provided for the purposes of illustration, and not intended to limit the scope of the present disclosure. For persons having ordinary skills in the art, multiple variations and modifications may be made under the teachings of the present disclosure. For example, the processing engine <NUM> may be integrated into the user terminal <NUM>.

<FIG> is a schematic diagram illustrating exemplary hardware and/or software components of an exemplary computing device according to some embodiments of the present disclosure. In some embodiments, the server <NUM> may be implemented on the computing device <NUM> shown in <FIG>. For example, the processing engine <NUM> may be implemented on the computing device <NUM> and configured to perform functions of the processing engine <NUM> disclosed in this disclosure.

The computing device <NUM> may be used to implement any component of the system <NUM> as described herein. For example, the processing engine <NUM> may be implemented on the computing device <NUM>, via its hardware, software program, firmware, or a combination thereof. Although only one such computer is shown, for convenience, the computer functions relating to image processing as described herein may be implemented in a distributed fashion on a number of similar platforms to distribute the processing load.

The computing device <NUM>, for example, may include COM ports <NUM> connected to and from a network connected thereto to facilitate data communications. The computing device <NUM> may also include a processor (e.g., the processor <NUM>), in the form of one or more processors (e.g., logic circuits), for executing program instructions. For example, the processor <NUM> may include interface circuits and processing circuits therein. The interface circuits may be configured to receive electronic signals from a bus <NUM>, wherein the electronic signals encode structured data and/or instructions for the processing circuits to process. The processing circuits may conduct logic calculations, and then determine a conclusion, a result, and/or an instruction encoded as electronic signals. Then the interface circuits may send out the electronic signals from the processing circuits via the bus <NUM>.

The exemplary computing device may further include program storage and data storage of different forms including, for example, a disk <NUM>, and a read-only memory (ROM) <NUM>, or a random-access memory (RAM) <NUM>, for various data files to be processed and/or transmitted by the computing device. The exemplary computing device may also include program instructions stored in the ROM <NUM>, RAM <NUM>, and/or another type of non-transitory storage medium to be executed by the processor <NUM>. The methods and/or processes of the present disclosure may be implemented as the program instructions. The computing device <NUM> may also include an I/O component <NUM>, supporting input/output between the computer and other components. The computing device <NUM> may also receive programming and data via network communications.

Merely for illustration, only one processor is illustrated in <FIG>. Multiple processors <NUM> are also contemplated; thus, operations and/or method steps performed by one processor <NUM> as described in the present disclosure may also be jointly or separately performed by the multiple processors. For example, if in the present disclosure the processor <NUM> of the computing device <NUM> executes both step A and step B, it should be understood that step A and step B may also be performed by two different processors <NUM> jointly or separately in the computing device <NUM> (e.g., a first processor executes step A and a second processor executes step B or the first and second processors jointly execute steps A and B).

<FIG> is a schematic diagram illustrating an exemplary terminal device according to some embodiments of the present disclosure. In some embodiments, the user terminal <NUM> may be implemented on the terminal device <NUM> shown in <FIG>. The terminal device <NUM> may be a mobile device, such as a mobile phone of a passenger or a driver, a built-in device on a vehicle driven by the driver. As illustrated in <FIG>, the terminal device <NUM> may include a communication platform <NUM>, a display <NUM>, a graphic processing unit (GPU) <NUM>, a central processing unit (CPU) <NUM>, an I/O <NUM>, a memory <NUM>, and a storage <NUM>. In some embodiments, any other suitable component, including but not limited to a system bus or a controller (not shown), may also be included in the terminal device <NUM>.

In some embodiments, an operating system <NUM> (e.g., iOS™, Android™, Windows Phone™, etc.) and one or more Apps (applications) <NUM> may be loaded into the memory <NUM> from the storage <NUM> in order to be executed by the CPU <NUM>. User interactions may be achieved via the I/O <NUM> and provided to the server <NUM> and/or other components of the system <NUM> via the network <NUM>. The terminal device <NUM> may transmit/receive data related to the image to be processed via the communication platform <NUM>. For example, the terminal device <NUM> may receive the position and the lane line category of one or more determined lane lines from the server <NUM> via the communication platform <NUM>.

<FIG> is a block diagram illustrating an exemplary detection device according to some embodiments of the present disclosure. As shown in <FIG>, the lane line detection device <NUM> may include a processor <NUM> and a storage <NUM>. One or more computer programs may be stored in the storage <NUM>, and the processor <NUM> may be used to execute the one or more computer programs to implement a lane line detection method.

In some embodiments, the lane line detection method may include obtaining an image and determining a semantic category for each of a plurality of pixels in the image. The lane line detection method may further include determining one or more pixel sets based on the plurality of pixels according to a predetermined rule. Each of the one or more pixel sets may include one or more pixels of a same semantic category. The lane line detection method may further include performing a fitting operation to obtain one or more fitting lines corresponding to one or more lane lines.

In some embodiments, the fitting operation may be an iterative fitting operation on each of the one or more pixels in the pixel set. In each iteration, it may be determined whether there are one or more erroneous pixels that satisfy a predetermined condition in a pixel set. In response to a determination that there are one or more erroneous pixels that satisfy the predetermined condition, the one or more erroneous pixels may be removed from the pixel set.

In some embodiments, a binarization operation may be performed to obtain binary information of each of the plurality of pixels. A lane line category of a lane line may be determined at least based on the binary information of the one or more pixels in a pixel set.

In some embodiments, the trained semantic segmentation network may include one or more convolutional layers and one or more deconvolutional layers. The one or more convolutional layers may be configured for a depth-wise separable convolution. For example, the trained semantic segmentation network may be constructed based on a Mobilenet and a fully convolutional network (FCN). More details regarding the lane line detection method may be found elsewhere in the present disclosure, for example, in <FIG>, <FIG>, and <FIG>.

In some embodiments, the one or more computer programs can be stored in a computer storage medium. According to another aspect of the present disclosure, a computer storage medium is provided.

<FIG> is a block diagram illustrating an exemplary computer storage medium according to some embodiments of the present disclosure. As shown in <FIG>, one or more computer programs may be stored in the computer storage medium <NUM>. When the one or more computer programs <NUM> are executed by the processor, the lane line detection method may be implemented.

The computer storage medium <NUM> may include a Universal Serial Bus (USB) flash disk, a mobile hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, or it can be a server (e.g., the server <NUM> in <FIG>) that stores the one or more computer programs <NUM>. The server may send the one or more stored computer programs to other devices, or the server may execute the one or more stored computer programs. In some embodiments, the computer storage medium <NUM> may be a combination of physical entities, such as a plurality of servers, a server, with a storage, or a storage with a mobile hard disk.

<FIG> is a block diagram illustrating an exemplary processing engine according to some embodiments of the present disclosure. In some embodiments, the processing engine <NUM> may be implemented on the server <NUM>. The processing engine <NUM> may communicate with a storage medium (e.g., the storage device <NUM> of the system <NUM>, and/or the storage <NUM> of the terminal device <NUM>), and may execute instructions stored in the storage medium. In some embodiments, the processing engine <NUM> may include an obtaining module <NUM>, a semantic category determination module <NUM>, a pixel set determination module <NUM>, a fitting module <NUM>, and a lane line determination module <NUM>.

The obtaining module <NUM> may obtain data from one or more components of the system <NUM>. In some embodiments, the obtaining module <NUM> may obtain an image. In some embodiments, the image may be associated with a scene on the road, and the image may also be referred to as a "road image". The obtaining module <NUM> may obtain the image from an acquisition device (e.g., the acquisition device <NUM> in <FIG>). For instance, the acquisition device may be a camera installed near the road or on a moving car. For example, the camera may be a surveillance camera attached to or as part of a street lamp or a road sign. The camera may acquire the information on the road (e.g., information associated with a scene on the road) at predetermined time intervals (e.g., <NUM> second, <NUM> second, <NUM> second, or the like).

The semantic category determination module <NUM> may determine a semantic category for each of a plurality of pixels in the road image. The semantic category determination module <NUM> may determine, for each of a plurality of pixels in the image. a semantic category using a trained semantic segmentation network. In some embodiments, the semantic category determination module <NUM> may determine the semantic category of each pixel of a region of interest (ROI) in the image. The ROI may include the road and objects on the road (e.g., vehicles, lane lines). In some embodiments, the semantic category determination module <NUM> may input the image into the trained semantic segmentation network. A semantic image may be outputted by the trained semantic segmentation network, and thus the semantic category of each pixel can be obtained. In some embodiments, the trained semantic segmentation network may identify whether a pixel corresponds to a lane line or a background. In some embodiments, the semantic category determination module <NUM> may further perform a binarization operation to obtain binary information of one or more pixels and determine a lane line category of a lane line corresponding to the pixel (e.g., whether the pixel corresponds to a dashed line or a solid line). In some embodiments, the semantic segmentation network may determine the lane line category of the lane line corresponding to the pixel. In some embodiments, the lane line in the image may be presented in red or green, the other portions may be rendered in black as the background. For example, the solid line may be displayed in red and the dashed line may be displayed in green. That is, the pixel value may be processed (e.g., by assigning different pixel values to pixels of different semantic categories), and the semantic category may be represented by the pixel value.

In some embodiments, the semantic segmentation network in the present disclosure may use a depth-wise separable convolution structure to extract one or more features of the images to obtain one or more feature convolutional layers (also referred to as "convolutional layers") that include the one or more features of the image. The one or more feature convolutional layers are then processed using a full convolution network (FCN) to obtain the semantic category for each pixel in the image.

The pixel set determination module <NUM> may determine one or more pixel sets based on the plurality of pixels in the road image. The pixel set determination module <NUM> may determine one or more pixel sets based on the plurality of pixels according to a predetermined rule. Each of the one or more pixel sets may include one or more pixels of a same semantic category. After obtaining the semantic category of each of the plurality of pixels in the image, the detection device may select one or more pixels of the same semantic category and filter the one or more pixels according to a predetermined rule, and the pixels may be combined into a pixel set. Each of the obtained pixel sets may correspond to a lane line. The predetermined rule is set so that the pixels that are relatively close can be combined in a pixel set. For example, the predetermined rule may include that an absolute value of a difference between coordinates of a pixel and a first pixel is less than or equal to a preset difference value. The coordinates may be an abscissa value and/or an ordinate value. Similarly, in other embodiments, the predetermined rule may also include other rules, and different rules may be set according to the requirements of the designer, and details are not described herein.

The fitting module <NUM> may perform a fitting operation to obtain one or more fitting lines corresponding to the one or more pixel sets. In some embodiments, the fitting module <NUM> may perform a fitting operation on each pixel set to obtain a fitting line as a lane line, and the fitting line may be a straight fitting line or a fitting curve. Generally, each pixel set may correspond to a lane line, and a fitting operation may be performed on each pixel set so that the corresponding lane line is determined. In some embodiments, the fitting operation may be an iterative fitting operation. The fitting module <NUM> may determine whether there are one or more erroneous pixels in each of the one or more pixel sets. In response to a determination that there are one or more erroneous pixels in a pixel set, the detection device may remove the one or more erroneous pixels from the pixel set. For example, the fitting module <NUM> may determine, for each of one or more pixels in the pixel set, a first difference between a fitted coordinate and an actual coordinate associated with the pixel. The actual coordinates of the pixels in the pixel set may be obtained, and a fitted coordinate of each pixel may be determined according to the estimated line equation. The first difference between the fitted coordinate (e.g., a fitted y-axis coordinate) and the actual coordinate (e.g., an actual y-axis coordinate) associated with the pixel may be determined and compared with the first threshold.

If the first difference is less than or equal to the first threshold, it means that the pixel is not far from the fitting line. The pixel may be considered as a normal pixel in a lane line, and no further processing may be needed for the pixel. If the first difference is greater than the first threshold, it means that the pixel is located relatively far from the fitting line. The pixel may be determined as an interfering pixel (i.e., an erroneous pixel). The fitting module <NUM> may remove one or more erroneous pixels of which the first difference is greater than a first threshold to update the pixel set. The fitting module <NUM> may re-perform the linear fitting operation on the updated pixel set. if for each of the one or more fitting parameters, a second difference between the fitting parameters estimated in two consecutive iterations is less than or equal to a second threshold, the fitting module <NUM> may stop updating the pixel set. The second difference may relate to a termination criterion for the iterative fitting operation. When the termination criterion is satisfied, the fitting module <NUM> may stop the iterations. For instance, the termination criterion may include that the second difference is less than or equal to the second threshold. In some embodiments, the second difference may be a difference value between the fitting parameters estimated in two consecutive iterations. In some embodiments, the second difference may be a rate of change between the fitting parameters estimated in two consecutive iterations.

The lane line determination module <NUM> may determine a position and a lane line category of the lane line. In some embodiments, the lane line determination module <NUM> may determine a position of one or more lane lines based on the fitting line. In some embodiments, the lane line determination module <NUM> may determine a lane line category of the lane line, for example, based on the semantic category of the pixel set corresponding to the lane line and/or binary information of the lane line. the trained semantic segmentation network may identify whether a pixel corresponds to a lane line or the background (i.e., the portion of the image that does not include any lane lines) without identifying the lane line category corresponding to the lane line. In some embodiments, the trained segmentation network may identify whether a pixel corresponds to a lane line or the background and a reference lane line category corresponding to the lane line. If the fitting parameters of a pixel set are not similar to any fitting parameters (i.e., a difference between two fitting parameters are less than a third threshold) of other pixel sets, the reference lane line category may be determined as the lane line category of a lane line corresponding to the pixel set. If at least two pixel sets of the same semantic category have similar fitting parameters, the lane line determination module <NUM> may combine the at least two pixel sets. The lane line determination module <NUM> may determine the lane line category associated with the combined pixel set based at least on the binary information of each of the one or more pixels in the combined pixel set.

In some embodiments, the lane line determination module <NUM> may perform a binarization operation to obtain binary information of each of the plurality of pixels. In some embodiments, the lane line determination module <NUM> may perform an edge extraction operation on the plurality of pixels in the image to obtain edge information associated with each of the plurality of pixels. The edge information associated with a pixel may indicate whether the pixel corresponds to a portion of an edge in the image. For instance, the lane line determination module <NUM> may use a Sobel operator to perform the edge extraction operation. The lane line determination module <NUM> may further determine, based on the edge information, the binary information of each of the plurality of pixels. The binary information may include the pixel value of a pixel after the binarization operation. For instance, if the pixel corresponds to a portion of an edge in the image, the pixel value of the pixel may be determined as <NUM>, and if the pixel does not correspond to a portion of an edge, the pixel value of the pixel may be determined as <NUM>. The lane line determination module <NUM> may determine, for each of the plurality of pixels, a searching range based on a fitted coordinate of the pixel. The lane line determination module <NUM> may traverse, for each of the one or more pixel sets, the one or more pixels in the pixel set and record a reference number count of reference pixels. If the reference number count is greater than a count threshold, the lane line determination module <NUM> may determine that the semantic category of the combined pixel set is the dashed line. If the reference number count is less than or equal to a count threshold, the lane line determination module <NUM> may determine that the semantic category of the combined pixel set is the solid line.

The modules in <FIG> may be connected to or communicate with each other via a wired connection or a wireless connection. The wired connection may include a metal cable, an optical cable, a hybrid cable, or the like, or a combination thereof. The wireless connection may include a Local Area Network (LAN), a Wide Area Network (WAN), a Bluetooth, a ZigBee, a Near Field Communication (NFC), or the like, or a combination thereof. In some embodiments, two or more of the modules may be combined into a single module, and any one of the modules may be divided into two or more units.

<FIG> is a flowchart illustrating an exemplary process for lane detection according to some embodiments of the present disclosure. In some embodiments, the process <NUM> may be executed by the system <NUM>. For example, the process <NUM> may be implemented as a set of instructions (e.g., an application) stored in the storage (e.g., ROM <NUM> or RAM <NUM> of the computing device <NUM>). The detection device <NUM>, the processing engine <NUM> and/or modules in <FIG> may execute the set of instructions, and when executing the instructions, these devices and/or modules may be configured to perform the process <NUM>. The operations of the illustrated process <NUM> presented below are intended to be illustrative. In some embodiments, the process <NUM> may be accomplished with one or more additional operations not described, and/or without one or more of the operations discussed. Additionally, the order in which the operations of the process <NUM> as illustrated in <FIG> and described below is not intended to be limiting.

In <NUM>, the processing engine <NUM> (e.g., the obtaining module <NUM>) may obtain an image. In some embodiments, the image may be associated with a scene on the road, and the image may also be referred to as a "road image". One or more road marks may be identified from the road image according to operations <NUM>-<NUM>. The one or more road marks may have different patterns or shapes. For instance, the one or more road marks may include a lane line, a speed limit tag painted on the road, a pedestrian crossing, a lane direction mark that indicates a direction toward which vehicles on this lane can move or the like, or any combination thereof. For illustration purposes, the following description mainly relates to the detection of lane lines. But it should be understood that the mechanism disclosed herein may be applied to various road marks.

In some embodiments, a detection device (e.g., the detection device <NUM>) may obtain the image. The detection device may be communicatively connected to an acquisition device (e.g., the acquisition device <NUM> in <FIG>). For instance, the acquisition device may be a camera installed near the road or on a moving car. For example, the camera may be a surveillance camera attached to or as part of a street lamp or a road sign. The camera may acquire the information on the road (e.g., information associated with a scene on the road) at predetermined time intervals (e.g., <NUM> second, <NUM> second, <NUM> second, or the like). The camera may use the acquired information as the image or synthesize the acquired information into the image, and then send the image to the detection device.

In some embodiments, the detection device may be connected to an external storage device, and the external storage device may be a mobile hard disk, a floppy disk drive, a Universal Serial Bus (USB) flash disk or an optical disk drive, etc. The external storage device may store the image, and the detection device may obtain the image directly from the external storage device.

<FIG> is an exemplary image obtained by the system <NUM> according to some embodiments of the present disclosure. The image may be obtained by a surveillance camera mounted, for example, on a road lamp or a street sign. As shown in <FIG>, the image presents two cars running on different lanes on a road. The image includes two solid lane lines <NUM> and two dash lane lines <NUM>. Theses lane lines may be identified according to the process <NUM>.

In <NUM>, the processing engine <NUM> (e.g., the semantic category determination module <NUM>) may determine, for each of a plurality of pixels in the image. a semantic category using a trained semantic segmentation network. In some embodiments, the processing engine <NUM> may determine the semantic category of each pixel of the image. In some embodiments, the processing engine <NUM> may determine the semantic category of each pixel of a region of interest (ROI) in the image. The ROI may include the road and objects on the road (e.g., vehicles, lane lines).

In some embodiments, the processing engine <NUM> may input the image into the trained semantic segmentation network. A semantic image may be outputted by the trained semantic segmentation network, and thus the semantic category of each pixel can be obtained. In some embodiments, the trained semantic segmentation network may identify whether a pixel corresponds to a lane line or a background. In some embodiments, the processing engine <NUM> may further perform a binarization operation to obtain binary information of one or more pixels and determine a lane line category of a lane line corresponding to the pixel (e.g., whether the pixel corresponds to a dashed line or a solid line). In some embodiments, the trained semantic segmentation network may determine the lane line category of the lane line corresponding to the pixel. In some embodiments, the lane line in the image may be presented in red or green, the other portions may be rendered in black as the background. For example, the solid line may be displayed in red and the dashed line may be displayed in green. That is, the pixel value may be designated (e.g., by assigning different pixel values to pixels of different semantic categories), and the semantic category may be represented by the pixel value.

<FIG> is an exemplary semantic image according to some embodiments of the present disclosure. In some embodiments, the two lines <NUM> may be displayed in green in the semantic image, indicating that the semantic category of each pixel on the two lines <NUM> is a dashed lane line. In some embodiments, the two lines <NUM> may be displayed in red in the semantic image, indicating that the semantic category of each pixel on the two lines <NUM> is a solid lane line.

In some embodiments, the trained semantic segmentation network may include a convolutional neural network (CNN), a fully convolutional network (FCN), SegNet, U-Net, or the like, or any combination thereof. In some embodiments, the trained semantic segmentation network in the present disclosure may use a depth-wise separable convolution structure to extract one or more features of the images to obtain one or more feature convolutional layers (also referred to as "convolutional layers") that include the one or more features of the image. The one or more feature convolutional layers are then processed using the FCN to obtain the semantic category for each pixel in the image. In some embodiments, the depth-wise separable convolution structure may include at least a part of the Mobilenet, the Xception, and/or other convolution structures. For example, the semantic segmentation network may be constructed based on the Mobilenet and the FCN. More details regarding the structure of the semantic segmentation network may be found elsewhere in the present disclosure, for example, in <FIG> and the description thereof.

In some embodiments, before operation <NUM>, the method for lane detection may further include: obtaining an image size corresponding to the trained semantic segmentation network, and preprocess the image so that the input size of the image is consistent with the image size used for training the semantic segmentation network.

In some embodiments, the processing engine <NUM> may obtain the trained semantic segmentation network from the storage device <NUM>. In some embodiments, a semantic segmentation network may be trained using a plurality of training samples. For example, each of the plurality of training samples may include a sample image and one or more labels corresponding to one or more pixels in the sample image. A label may indicate a semantic category of the pixel that is examined by a user. In some embodiments, the label may indicate whether the pixel corresponds to the background or a lane line. In some embodiments, the label may indicate that the pixel corresponds to the background, a dashed lane line, a solid lane line, or other types of lane lines.

In <NUM>, the processing engine <NUM> (e.g., the pixel set determination module <NUM>) may determine one or more pixel sets based on the plurality of pixels according to a predetermined rule. Each of the one or more pixel sets may include one or more pixels of a same semantic category.

After obtaining the semantic category of each of the plurality of pixels in the image, the detection device may select one or more pixels of the same semantic category and filter the one or more pixels according to a predetermined rule, and the pixels may be combined into a pixel set. Each of the obtained pixel sets may correspond to a lane line. The predetermined rule is set so that the pixels that are relatively close can be combined in a pixel set.

For example, the predetermined rule may include that an absolute value of a difference between coordinates of a pixel and a first pixel is less than or equal to a preset difference value. The coordinates may be an abscissa value and/or an ordinate value. Similarly, in other embodiments, the predetermined rule may also include other rules, and different rules may be set according to the requirements of the designer, and details are not described herein.

In some embodiments, according to operation <NUM>, the detection device may obtain one or more pixel sets corresponding to a semantic category. For instance, the detection device may determine one or more pixel sets corresponding to one or more lane lines, respectively. As another example, the detection device may determine one or more pixel sets corresponding to one or more dashed lane lines and/or one or more solid lane lines, respectively.

In <NUM>, the processing engine <NUM> (e.g., the fitting module <NUM>) may perform, for each of the one or more pixel sets, a fitting operation to obtain a fitting line corresponding to a lane line.

In some embodiments, the detection device may perform a fitting operation on each pixel set to obtain a fitting line as a lane line, and the fitting line may be a fitting straight line, a fitting curve, or the like. Generally, each pixel set may correspond to a lane line, and a fitting operation may be performed on each pixel set so that the corresponding lane line is obtained, that is, the detection of the lane line is implemented. In some embodiments, the fitting operation may be an iterative fitting operation. The detection device (e.g., the processing engine <NUM>) may determine whether there are one or more erroneous pixels in each of the one or more pixel sets. In response to a determination that there are one or more erroneous pixels in a pixel set, the detection device may remove the one or more erroneous pixels from the pixel set.

In some embodiments, the detection device may process the image using a pre-trained (i.e., trained in advance before usage) semantic segmentation network, thereby acquiring the semantic category of each pixel in the image and determining the pixel set representing the lane line according to the semantic category. The detection device may further perform a fitting operation on the pixel set, thereby implementing the detection of lane lines. Specifically, in the semantic segmentation network, the depth-wise separable convolution structure may be used to extract the image features, obtain the feature convolution layer of the image, and then use the full convolution network to process the feature convolution layer to obtain the semantic category of each pixel in the image. Through the Mobilenet-FCN, the number of parameters in the image feature extraction can be reduced, thereby reducing the amount of calculation, and by concatenating the fully connected layer of the Mobilenet into the deconvolution process, the image information can be restored to achieve the purpose of pixel classification (i.e., determining the semantic category of the pixel), thereby improving the accuracy of the pixel classification. After the pixels are classified, the pixels having the same semantic category and satisfying the predetermined rule may be added in a pixel set to obtain one or more pixel sets. A fitting operation may be performed for each pixel set to obtain a fitting line as a lane line.

In some embodiments, the processing engine <NUM> (e.g., the lane line determination module <NUM>) may determine a position of one or more lane lines based on the fitting line. In some embodiments, the processing engine <NUM> may determine a lane line category of the lane line, for example, based on the semantic category of the pixel set corresponding to the lane line and/or binary information of the lane line. Such information related to one or more lane lines may be used in a self-driving technique, traffic violation monitoring, and/or other fields. For instance, in the self-driving technique, a vehicle may follow the lane line while moving. As another example, for traffic violation monitoring, if a vehicle is found to have changed lanes in an area where lane changing is prohibited, it may be determined that the vehicle has violated a traffic rule.

It should be noted that the above description is merely provided for the purposes of illustration, and not intended to limit the scope of the present disclosure.

<FIG> is a schematic diagram illustrating an exemplary semantic segmentation network according to some embodiments of the present disclosure. Merely by way of example, the semantic segmentation network is constructed based on the Mobilenet and the FCN. A part of the semantic segmentation network corresponding to the Mobilenet may be used to extract image features of the image (i.e., the image inputted to the semantic segmentation network). Compared with traditional 3D convolution, the Mobilenet uses depth-wise (depth-wise separable convolution structure) convolution to process the image. The depth-wise separable convolution structure can effectively reduce the redundant expression of the convolution. The parameters extracted by the Mobilenet are reduced to <NUM>/<NUM> as compared with the traditional 3D convolution, and the accuracy is maintained. Since the number count of parameters has dropped significantly, the Mobilenet may be embedded in various platforms (e.g., a mobile device such as a smartphone, an on-board device of a vehicle).

Specifically, the image is input to the part corresponding to the Mobilenet, and a plurality of feature convolutional layers may be obtained after N*N convolution. After applying the convolutional layers, the Mobilenet uses several fully connected layers to connect with the convolutional layers, and information of the feature convolutional layer is mapped into a feature vector of a fixed length.

The FCN restores the size of the feature vector of the Mobilenet to the original image size by deconvolution and classifies each pixel. If the deconvolution is performed on the last fully connected layer of the Mobilenet, it may cause the image to lose much semantic information, while a few layers before the last convolutional layer of the Mobilenet retain much information such as image edges, colors, contours, or other information, or a combination thereof. Therefore, the semantic segmentation network connects a few layers before the last convolutional layer of the Mobilenet to the deconvolutional layers of the full convolution network, thereby restoring more image information and obtaining the semantic category of each pixel in the image. For instance, one or more convolutional layers and one or more deconvolutional layers may be connected via skip connections. That is, each of the one or more convolutional layers may be connected with a deconvolutional layer that is arranged after a few layers after the convolutional layer. For instance, as shown in <FIG>, the Conv3 layer (a convolutional layer) may be connected with the Crop <NUM> layer that is configured to change a size of the feature vector obtained by Conv3 layer. The Crop2 layer may be connected to a Deconv2 layer (a deconvolutional layer). The Deconv2 layer may perform a deconvolution operation on the feature vector. Thus more information of the input image may be retained. The semantic segmentation network may output a semantic image of the same size as the input image. The semantic category of each of a plurality of pixels of the semantic image may be determined.

It should be noted that the above description is merely provided for the purposes of illustration, and not intended to limit the scope of the present disclosure. For example, the number count of convolutional layers, crop layers and/or convolutional layers may be different from what is shown in <FIG>.

<FIG> is a flowchart illustrating an exemplary process for determining one or more pixel sets according to some embodiments of the present disclosure. In some embodiments, the process <NUM> may be executed by the system <NUM>. For example, the process <NUM> may be implemented as a set of instructions (e.g., an application) stored in the storage (e.g., ROM <NUM> or RAM <NUM> of the computing device <NUM>). The detection device <NUM>, the processing engine <NUM> and/or modules in <FIG> may execute the set of instructions, and when executing the instructions, these devices and/or modules may be configured to perform the process <NUM>. The operations of the illustrated process <NUM> presented below are intended to be illustrative. In some embodiments, the process <NUM> may be accomplished with one or more additional operations not described, and/or without one or more of the operations discussed. Additionally, the order in which the operations of the process <NUM> as illustrated in <FIG> and described below is not intended to be limiting.

In <NUM>, the processing engine <NUM> (e.g., the pixel set determination module <NUM>) may traverse a plurality of pixels in the image.

In some embodiments, a rectangular coordinate system may be determined based on the semantic image or the road image. Since the semantic image and the road image are of the same image size, the semantic image and the road image may utilize the same rectangular coordinate system. For instance, the coordinates of the pixel in the lower left corner of the road image may be designated as the origin; a lateral direction of the road image may be designated as the Y axis, and a longitudinal direction of the road image may be designated as the X axis. The plurality of pixels in the road mage may be traversed according to a predetermined direction, for example, from left to right and/or from top to bottom, to determine the semantic category of each of the plurality of pixels. In some embodiments, the plurality of pixels in the semantic image may be traversed, from left to right and from top to bottom, to detect the pixel value of the pixel. The traversal may be performed sequentially, and the pixel set may be successfully generated based on the plurality of pixels.

In <NUM>, the processing engine <NUM> (e.g., the pixel set determination module <NUM>) may determine whether there are one or more reference pixel sets that have the same semantic category as the pixel.

As used herein, the term "reference pixel set" refers to a pixel set generated based on the semantic image during the determination of the one or more pixel sets corresponding to one or more lane lines. One or more pixels may be added into a reference pixel set. After the process <NUM> is completed, one or more reference pixel sets may be designated as the one or more pixel sets corresponding to one or more lane lines. In some embodiments, for a current pixel traversed sequentially, the semantic category of the current pixel may be obtained, and the processing engine <NUM> may determine whether there are one or more reference pixel sets that have the same semantic category as the current pixel. If there are no reference pixels that have the same semantic category as the current pixel, the processing engine <NUM> may proceed to step operation <NUM>; if there are one or more reference pixels that have the same semantic category as the current pixel, the processing engine <NUM> may proceed to operation <NUM>.

In <NUM>, the processing engine <NUM> (e.g., the pixel set determination module <NUM>) may generate a new reference pixel set for the pixel.

For example, when it is detected that the pixel value of the current pixel is red (<NUM>, <NUM>, <NUM>), and that there are no reference pixel sets that have the same semantic category as the current pixel, the processing engine <NUM> may generate a new reference pixel set. The current pixel may be added in the new reference pixel set as the first pixel.

In other words, when the processing engine <NUM> traverses the plurality of pixels in the semantic image, a pixel that is green (<NUM>, <NUM>, <NUM>) or red (<NUM>, <NUM>, <NUM>) may be firstly detected. A container <NUM> (i.e., a reference pixel set) may be generated, and the number count of lane lines is increased from <NUM> to <NUM>. The coordinate values of the pixel and the corresponding semantic category may be saved in the container <NUM>. The container <NUM> can be represented by a pixel set or in other forms, and details are not described herein.

For example, when a pixel whose Red-Green-Blue (RGB) value is green (<NUM>, <NUM>, <NUM>) is detected for the first time, a green container <NUM> may be created, and the coordinate values of the pixel and the corresponding semantic category (dashed line) may be saved in the green container <NUM>.

In <NUM>, the processing engine <NUM> (e.g., the pixel set determination module <NUM>) may determine a minimum distance from one or more distances between the pixel and the one or more reference pixel sets.

In some embodiments, the processing engine <NUM> may determine a distance between the pixel and each of the one or more reference pixel sets that have the same semantic category as the current pixel. For example, the processing engine <NUM> may determine a distance between the pixel that is currently traversed and a pixel that is recently added into the reference pixel sets. The processing engine <NUM> may further determine a minimum distance from one or more distances between the pixel and the one or more reference pixel sets.

In some embodiments, the plurality of pixels in the image (i.e., the road image or the semantic image) may be traversed in a certain order, for example, from left to right and from top to bottom. One or more pixels may be added to a reference pixel set sequentially, and therefore, the distance between the current pixel and the reference pixel set may be the distance between the current pixel and the pixel stored in the pixel set, that is, the distance between the current pixel and the uppermost pixel in the container (i.e., a pixel that is most recently added into the reference pixel set). In some embodiments, the same principles may be employed if the semantic images are traversed in a different manner.

In <NUM>, the processing engine <NUM> (e.g., the pixel set determination module <NUM>) may compare the minimum distance with a distance threshold. The distance threshold may be a default value of the system <NUM> or may be determined by a user. For instance, the distance threshold may be <NUM>, <NUM>, <NUM>, etc..

In <NUM>, if the minimum distance is less than the distance threshold, the processing engine <NUM> (e.g., the pixel set determination module <NUM>) may add the pixel into a reference pixel set corresponding to the minimum distance.

If the minimum distance is less than the preset distance threshold, it indicates that the current pixel is mostly close to the recent pixel in the reference pixel set and that the current pixel may be considered as the pixel of the same lane line. Thus, the current pixel may be added to the reference pixel set.

For example, when a pixel whose pixel value is green (<NUM>, <NUM>, <NUM>) is detected again, if the minimum distance between the pixel and the recent pixel in the green container <NUM> is less than the distance threshold, the pixel may be added into the green container <NUM>.

In <NUM>, if the minimum distance is greater than the distance threshold, the processing engine <NUM> (e.g., the pixel set determination module <NUM>) may generate a new reference pixel set for the pixel.

If the minimum distance is greater than or equal to the distance threshold, it indicates that the current pixel is relatively far from the one or more reference pixel sets, and that the current pixel may correspond to another lane line that is different from the one or more lane lines corresponding to the one or more reference pixel sets. Therefore, instead of adding the current pixel to the one or more existing reference pixel sets, a new reference pixel set may be generated based on the current pixel.

For example, when a pixel with the RGB value of green (<NUM>, <NUM>, <NUM>) is detected for a second time, if it is determined that the minimum distance of the pixel and a recent pixel in the green container <NUM> is greater than or equal to the distance threshold, a new green container <NUM> may be created, the number of lane lines may be increased by <NUM>. The current pixel may be added into the green container <NUM>.

When traversing the plurality of pixels sequentially, operations <NUM>-<NUM> may be performed for each of the plurality of pixels. After the traversing process, one or more reference pixel sets may be designated as one or more pixel sets corresponding to one or more lane lines. The number count of pixel sets may indicate the number count of lane lines in the semantic image and the road image. Further, a fitting operation may be performed on a plurality of pixel sets. The position and a lane line category of each of the one or more lane lines (a dashed line or solid line) can be obtained.

In <NUM>, the processing engine <NUM> (e.g., the obtaining module <NUM>) may obtain an image.

In <NUM>, the processing engine <NUM> (e.g., the semantic category determination module <NUM>) may determine, for each of a plurality of pixels in the image, a semantic category using a trained semantic segmentation network.

In <NUM>, the processing engine <NUM> (e.g., the pixel set determination module <NUM>) may determine one or more pixel sets based on the plurality of pixels according to a predetermined rule, each of the one or more pixel sets including one or more pixels of a same semantic category.

Operations <NUM> to <NUM> may be performed in a similar manner as described in operations <NUM> to <NUM>.

In <NUM>, the processing engine <NUM> (e.g., the fitting module <NUM>) may determine, for each of the one or more pixel sets, whether there are one or more erroneous pixels that satisfy a predetermined condition by performing an iterative fitting operation on the one or more pixels in the pixel set. In some embodiments, the iterative fitting operation may include a plurality of iterations. In each iteration associated with the pixel set, the processing engine <NUM> may perform a fitting operation on coordinates of one or more pixels in the pixel set to obtain an estimated fitting line, and determine whether there are one or more erroneous pixels that satisfy the predetermined condition.

In <NUM>, in response to a determination that there are one or more erroneous pixels, the processing engine <NUM> (e.g., the fitting module <NUM>) may remove the one or more erroneous pixels from the pixel set to obtain a fitting line corresponding to a lane line. In some embodiments, in each iteration associated with the pixel set, the processing engine <NUM> may determine whether there are one or more erroneous pixels in the pixel set. In response to a determination that there are one or more erroneous pixels in the pixel set, the processing engine <NUM> may remove the one or more erroneous pixels in the pixel set to update the pixel set. In a next iteration, the processing engine <NUM> may re-perform a fitting operation on the updated pixel set and generate an estimated fitting line and determine whether there are one or more erroneous pixels in the updated pixel set. After a termination criterion for the iterative fitting operation is satisfied, the processing engine <NUM> may stop the iterations, and designate the updated pixel set in the final iteration as a pixel set corresponding to a lane line. More details regarding the iterative fitting operation may be found elsewhere in the present disclosure, for example, in <FIG> and the description thereof.

Each pixel set may include a plurality of pixels that form a lane line. In some embodiments, the semantic category of the pixel set may also indicate the lane line category of the lane line. By classifying the above pixel values, it is possible to recognize the lane line in various scenes. In some embodiments, problems such as interference lines or semantic category errors may occur due to factors such as light (e.g., strong light, weak light) and environment (e.g., when it is rainy). As compared to a traditional simple fitting operation, the iterative operation may help improve the accuracy of the lane line.

<FIG> is a schematic diagram illustrating an exemplary interfering line and an exemplary lane line according to some embodiments of the present disclosure. As shown in <FIG>, after a traditional fitting operation is performed, a portion of a side wall of the green belt is mistakenly determined as a pixel set corresponding to a lane line, which is actually an interfering line. If the processing engine <NUM> performs the iterative fitting operation, for example, as described in connection with <FIG> or <FIG>, the pixels corresponding to the interfering line may be determined as erroneous pixels and may be removed from the pixel set. Thus, the position of the lane line may be more accurately determined.

<FIG> is a flowchart illustrating an exemplary process for an iterative fitting operation according to some embodiments of the present disclosure. In some embodiments, the process <NUM> may be executed by the system <NUM>. For example, the process <NUM> may be implemented as a set of instructions (e.g., an application) stored in the storage (e.g., ROM <NUM> or RAM <NUM> of the computing device <NUM>). The detection device <NUM>, the processing engine <NUM> and/or modules in <FIG> may execute the set of instructions, and when executing the instructions, these devices and/or modules may be configured to perform the process <NUM>. The operations of the illustrated process <NUM> presented below are intended to be illustrative. In some embodiments, the process <NUM> may be accomplished with one or more additional operations not described, and/or without one or more of the operations discussed. Additionally, the order in which the operations of the process <NUM> as illustrated in <FIG> and described below is not intended to be limiting.

In <NUM>, the processing engine <NUM> (e.g., the fitting module <NUM>) may perform a linear fitting operation on a pixel set.

In some embodiments, a coordinate system may be established, for example, as described in connection with operation <NUM>. Merely by way of example, a linear fitting operation may be performed to obtain a straight fitting line, thereby obtaining a straight line equation for a fitted lane line.

Specifically, if (x, y) is the actual coordinates of the pixel in a certain pixel set, it is assumed that the line equation obtained by the linear fitting operation on the pixel set is y=k*x+b. k denotes an intercept of the fitting line, while b denotes a slope of the fitting line. In some embodiments, the loss function of the fitting line based on the line equation may be presented using the following equation (<NUM>) <MAT> where n denotes the number of pixels in the pixel set; i denotes a specific pixel in the pixel set; and yi is a fitted coordinate value determined based on the line equation and the actual coordinate xl. Since y=k*x+b, the loss function may also be expressed using the following equation (<NUM>) <MAT>.

The partial derivatives of the loss function may be determined, respectively, using the following equation (<NUM>) and equation (<NUM>): <MAT> and <MAT>.

Where, assuming that <MAT>, C = <MAT>, and <MAT>, then, the fitting parameter k (i.e., the intercept of a straight line) and the fitting parameter b (i.e., the slope of the straight line) in the straight line equation may be determined according to the following equation (<NUM>):, <MAT> and <MAT>.

In each iteration of the iterative fitting operation, an estimated fitting line is obtained based on the estimated values of k and b. In the semantic image, when the semantic category of two pixels are the same, and the positions of the two pixels are relatively close, the two pixels may be considered to be added in the same reference pixel set. But in fact, if one of the pixels is an erroneous pixel, it may affect the linear fitting result of the pixel set. As a result, the least square method may cause the straight line to tilt to the left or right side, and thus the accuracy of the fitting line may be decreased. Therefore, the lane line detection method of this embodiment may include removing the interfering points according to operations <NUM>-<NUM>, and iteratively updates the pixel set until the rate of change of the fitting parameter k and the fitting parameter b are less than corresponding parameter thresholds, respectively.

In <NUM>, the processing engine <NUM> (e.g., the fitting module <NUM>) may determine, for each of one or more pixels in the pixel set, a first difference between a fitted coordinate and an actual coordinate associated with the pixel.

The actual coordinates of the pixels in the pixel set may be obtained, and a fitted coordinate of each pixel may be determined according to the estimated line equation. The first difference between the fitted coordinate (e.g., a fitted y-axis coordinate) and the actual coordinate (e.g., an actual y-axis coordinate) associated with the pixel may be determined and compared with the first threshold.

If the first difference is less than or equal to the first threshold, it means that the pixel is not far from the fitting line. The pixel may be considered as a normal pixel in a lane line, and no further processing may be needed for the pixel. If the first difference is greater than the first threshold, it means that the pixel is located relatively far from the fitting line. The pixel may be determined as an interfering pixel (i.e., an erroneous pixel) that needs to be removed in operation <NUM>.

In <NUM>, the processing engine <NUM> (e.g., the fitting module <NUM>) may remove one or more erroneous pixels of which the first difference is greater than a first threshold to update the pixel set.

In <NUM>, the processing engine <NUM> (e.g., the fitting module <NUM>) may re-perform the linear fitting operation on the updated pixel set.

In some embodiments, the linear fitting operation may be re-performed on the updated pixel set to obtain an updated fitting parameter k and updated fitting parameter b. The updated fitting parameters k and b are compared with the fitting parameters k and b before the update. The rate of change of the fitting parameter k and the rate of change of the fitting parameter b may be obtained.

In <NUM>, if for each of the one or more fitting parameters, a second difference between the fitting parameters estimated in two consecutive iterations is less than or equal to a second threshold, the processing engine <NUM> (e.g., the fitting module <NUM>) may stop updating the pixel set. The second difference may relate to a termination criterion for the iterative fitting operation. When the termination criterion is satisfied, the processing engine <NUM> may stop the iterations. For instance, the termination criterion may include that the second difference is less than or equal to the second threshold. In some embodiments, the second difference may be a difference value between the fitting parameters estimated in two consecutive iterations. In some embodiments, the second difference may be a rate of change between the fitting parameters estimated in two consecutive iterations.

If the rate of change of the at least one fitting parameter is greater than or equal to the corresponding second threshold, it indicates that there are one or more erroneous pixels in the pixel set, which may affect the accuracy of the fitting line. Operations <NUM>-<NUM> may be repeated. If the rates of change of both of the fitting parameters (k and b) are less than the corresponding second thresholds, the iteration of the least squares method may be stopped.

In some embodiments, when the first difference of each of the plurality of pixels in the pixel set is less than the first threshold, and the second difference for the estimated fitting line is still greater than or equal to the second threshold, the value of the first threshold may be reduced. Then operations <NUM>-<NUM> may be repeated until the second difference is less than the second threshold. In the linear fitting process, the lane line detection method tests whether there are one or more erroneous pixels in the pixel set. If there are one or more erroneous pixels in the pixel set, the one or more erroneous pixels may be deleted or removed, thereby eliminating the influence of the one or more erroneous pixels. In this way, the accuracy of the fitting line is improved, thus improving the likelihood of correctly determining the position of the lane line.

It should be noted that the above description is merely provided for the purposes of illustration, and not intended to limit the scope of the present disclosure. For example, other termination criteria may also be adopted, which are not limited by the present disclosure.

In <NUM>, the processing engine <NUM> (e.g., the lane line determination module <NUM>) may perform a binarization operation to obtain binary information of each of the plurality of pixels. In some embodiments, the processing engine <NUM> may perform an edge extraction operation on the plurality of pixels in the image to obtain edge information associated with each of the plurality of pixels. The edge information associated with a pixel may indicate whether the pixel corresponds to a portion of an edge in the image. For instance, the processing engine <NUM> may use an edge extraction operator to perform the edge extraction operation. Exemplary edge extraction operators may include a Sobel operator, a Prewitt operator, a Robert operator, a Laplace operator, or the like, or any combination thereof. The processing engine <NUM> may further determine, based on the edge information, the binary information of each of the plurality of pixels. The binary information may include the pixel value of a pixel after the binarization operation. For instance, if the pixel corresponds to a portion of an edge in the image, the pixel value of the pixel may be determined as <NUM>, and if the pixel does not correspond to a portion of an edge, the pixel value of the pixel may be determined as <NUM>.

In <NUM>, the processing engine <NUM> (e.g., the lane line determination module <NUM>) may perform, for each of the one or more pixel sets, a fitting operation on the one or more pixels in the pixel set to obtain a fitting line corresponding to a lane. In some embodiments, operation <NUM> may be performed in a similar manner as described in connection with operation <NUM>.

In <NUM>, the processing engine <NUM> (e.g., the lane line determination module <NUM>) may determine a position of the lane line based on the fitting line. For instance, a coordinate system established based on the image. The position of the lane line may be determined based on an equation corresponding to the fitting line (i.e., the line equation).

In <NUM>, the processing engine <NUM> (e.g., the lane line determination module <NUM>) may determine, based at least on the binary information of the one or more pixels in the pixel set, a lane line category of the lane line. In some embodiments, in operation <NUM>, the trained semantic segmentation network may identify whether a pixel corresponds to a lane line or the background (i.e., the portion of the image that does not include any lane lines) without identifying the lane line category corresponding to the lane line. The lane line category of the lane line may be determined based at least on the binary information of each of the one or more pixels in the pixel set. In some embodiments, in operation <NUM>, the trained segmentation network may identify whether a pixel corresponds to a lane line or the background and a reference lane line category corresponding to the lane line. If the fitting parameters of a pixel set are not similar to any fitting parameters (i.e., a difference between two fitting parameters are greater than a third threshold) of other pixel sets, the reference lane line category may be determined as the lane line category of a lane line corresponding to the pixel set. If at least two pixel sets of the same semantic category have similar fitting parameters, the processing engine <NUM> may combine the at least two pixel sets. The processing engine <NUM> may determine the lane line category associated with the combined pixel set based at least on the binary information of each of the one or more pixels in the combined pixel set. More details regarding the determination of the lane line category may be found elsewhere in the present disclosure, for example, in the description in connection with <FIG> and/or <FIG>.

<FIG> is a flowchart illustrating an exemplary process for determining a fitting line corresponding to a combined pixel set according to some embodiments of the present disclosure. In some embodiments, the process <NUM> may be executed by the system <NUM>. For example, the process <NUM> may be implemented as a set of instructions (e.g., an application) stored in the storage (e.g., ROM <NUM> or RAM <NUM> of the computing device <NUM>). The detection device <NUM>, the processing engine <NUM> and/or modules in <FIG> may execute the set of instructions, and when executing the instructions, these devices and/or modules may be configured to perform the process <NUM>. The operations of the illustrated process <NUM> presented below are intended to be illustrative. In some embodiments, the process <NUM> may be accomplished with one or more additional operations not described, and/or without one or more of the operations discussed. Additionally, the order in which the operations of the process <NUM> as illustrated in <FIG> and described below is not intended to be limiting.

In <NUM>, the processing engine <NUM> (e.g., the fitting module <NUM>) may perform a fitting operation on each of the one or more pixel sets and store one or more fitting parameters. In some embodiments, due to an inappropriate setting for the values of the first threshold and/or the second thresholds, there may be a problem in which pixels of the same lane line are assigned to different pixel sets. Operation <NUM> may be performed in a similar manner as described, for example, in connection with operation <NUM>.

In <NUM>, the processing engine <NUM> (e.g., the lane line determination module <NUM>) may compare, for every two pixel sets of the one or more pixel sets, the fitting parameters and the semantic categories. In some embodiments, the processing engine <NUM> may firstly determine whether there are at least two pixel sets corresponding to the same semantic category. In response to a determination that there are at least two pixel sets of the same semantic category in the one or more pixel sets, the processing engine <NUM> may further compare the fitting parameters of the at least two pixel sets corresponding to the same semantic category. For instance, the processing engine <NUM> may compare the intercept of the fitting lines of the at least two pixel sets corresponding to the same semantic category. In some embodiments, the processing engine <NUM> may determine a difference between fitting parameters (e.g., intercepts) corresponding to the at least two pixel sets and compare the difference with a parameter difference threshold. In response to a determination that the difference is less than or equal to the parameter difference threshold, the processing engine <NUM> may proceed to operation <NUM>.

In <NUM>, the processing engine <NUM> (e.g., the lane line determination module <NUM>) may combine pixel sets of the same semantic category. A parameter difference between the fitting parameters of the pixel sets may be less than a parameter threshold.

In <NUM>, the processing engine <NUM> (e.g., the lane line determination module <NUM>) may perform a linear fitting operation on the combined pixel set to obtain a fitting line corresponding to the lane line. In some embodiments, operation <NUM> may be performed in a similar manner as described in connection with operation <NUM>. In some embodiments, a portion of a lane line may be invisible in the image, for example, due to a vehicle that shelters the portion of the lane line. As a result, a single lane line may be identified as two lane lines corresponding to two pixel sets. The combination of similar pixel sets may decrease error caused by this problem and further improve the accuracy of lane line detection.

<FIG> is a flowchart illustrating an exemplary process for determining a semantic category of the combined pixel set according to some embodiments of the present disclosure. In some embodiments, the process <NUM> may be executed by the system <NUM>. For example, the process <NUM> may be implemented as a set of instructions (e.g., an application) stored in the storage (e.g., ROM <NUM> or RAM <NUM> of the computing device <NUM>). The detection device <NUM>, the processing engine <NUM> and/or modules in <FIG> may execute the set of instructions, and when executing the instructions, these devices and/or modules may be configured to perform the process <NUM>. The operations of the illustrated process <NUM> presented below are intended to be illustrative. In some embodiments, the process <NUM> may be accomplished with one or more additional operations not described, and/or without one or more of the operations discussed. Additionally, the order in which the operations of the process <NUM> as illustrated in <FIG> and described below is not intended to be limiting.

In <NUM>, the processing engine <NUM> (e.g., the fitting module <NUM>) may perform a fitting operation on each of the one or more pixel sets and store one or more fitting parameters.

In <NUM>, the processing engine <NUM> (e.g., the lane line determination module <NUM>) may compare, for every two pixel sets of the one or more pixel sets, the fitting parameters and the semantic categories.

In <NUM>, the processing engine <NUM> (e.g., the lane line determination module <NUM>) may combine pixel sets of the same semantic category, of which a parameter difference between the fitting parameters of the pixel sets is less than a parameter threshold.

In <NUM>, the processing engine <NUM> (e.g., the lane line determination module <NUM>) may perform a linear fitting operation on the combined pixel set to obtain a fitting line corresponding to the lane line.

In some embodiments, operations <NUM> to operations <NUM> may be performed in a similar manner as described in connection with operations <NUM>-<NUM>. In some embodiments, the trained segmentation network may identify whether a pixel corresponds to a lane line or the background and a reference lane line category corresponding to the lane line. For example, the semantic category of the lane line may include the dashed lane line, the solid lane line, and the background. In some embodiments, due to factors such as the light and the environment, a portion of a lane line may be determined as a dashed line, and another portion of the lane line may be determined as a solid line.

<FIG> is a schematic diagram illustrating exemplary pixels of an erroneous semantic category according to some embodiments of the present disclosure. As shown in <FIG>, pixels corresponding to a dashed line may be classified to generate a pixel set <NUM> corresponding to a dashed lane line, a pixel set <NUM> corresponding to a solid lane line, and a pixel set <NUM> corresponding to a dashed lane line. Operations <NUM> to <NUM> may be performed to improve the accuracy of the lane line category corresponding to the lane line.

In <NUM>, the processing engine <NUM> (e.g., the lane line determination module <NUM>) may perform a binarization operation to obtain binary information of each of the plurality of pixels. In some embodiments, the binarization operation may be performed on the same road image that is inputted to the trained semantic segmentation network. In some embodiments, the processing engine <NUM> may perform an edge extraction operation on the road image (e.g., by using the Sobel operator) before performing the binarization operation.

<FIG> is a schematic diagram illustrating an exemplary binary image according to some embodiments of the present disclosure. For example, the pixel value may be determined as <NUM> or <NUM>. The road image may present only two colors-black and white. Pixels of the white color may correspond to one or more edges in the binary image.

In <NUM>, the processing engine <NUM> (e.g., the lane line determination module <NUM>) may determine, for each of the plurality of pixels, a searching range based on a fitted coordinate of the pixel. In some embodiments, the processing engine <NUM> may obtain an X-axis coordinate x. The processing engine <NUM> may further determine a fitted y-axis coordinate y. The searching range may be determined as (y i - w, y i + w), where w denotes a parameter related to a width of the lane line corresponding to the pixel set. In some embodiments, the width of different portions of the lane line may look different in the road image due to factors such as distortion caused by the camera. The processing engine <NUM> may determine the maximum width of the lane line in the road image and determine the maximum width as the value of w.

In <NUM>, the processing engine <NUM> (e.g., the lane line determination module <NUM>) may traverse, for each of the one or more pixel sets, the one or more pixels in the pixel set and record a reference number count of reference pixels. If there are no white pixels within the searching range of each of the one or more pixels in a pixel set, the pixel may be determined as a reference pixel. Pixels corresponding to at least a portion of an edge may be referred to as "edge pixels". As used herein, the white pixels are a type of edge pixels whose pixel values are determined as <NUM>. For instance, if there are no white pixels within the searching range of a pixel, the pixel may be determined as a reference pixel, and a count value of the pixel may be increased by <NUM>. The processing engine <NUM> may determine the reference number count of pixels based on a sum of the count value corresponding to each pixel in the pixel set.

In <NUM>, if the reference number count is greater than a count threshold, the processing engine <NUM> (e.g., the lane line determination module <NUM>) may determine that the semantic category of the combined pixel set is the dashed line. Merely by way of example, the count threshold may be <NUM>, <NUM>, <NUM>, <NUM>, or the like. The count threshold may be a default value of the system <NUM> and may be adjustable.

In <NUM>, if the reference number count is less than or equal to the count threshold, the processing engine <NUM> (e.g., the lane line determination module <NUM>) may determine that the semantic category of the combined pixel set is the solid line. In some embodiments, operations <NUM> to <NUM> may be performed to correct an error that may occur in the determination of the semantic category of the plurality of pixels determined by the trained segmentation network. Thus, the accuracy of the lane line category corresponding to the one or more lane lines corresponding to the one or more pixel sets may be improved.

Having thus described the basic concepts, it may be rather apparent to those skilled in the art after reading this detailed disclosure that the foregoing detailed disclosure is intended to be presented by way of example only and is not limiting.

" Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable media having computer-readable program code embodied thereon.

Such a propagated signal may take any of a variety of forms, including electromagnetic, optical, or the like, or any suitable combination thereof.

Computer program code for carrying out operations for aspects of the present disclosure may be written in a combination of one or more programming languages, including an object oriented programming language such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB. NET, Python or the like, conventional procedural programming languages, such as the "C" programming language, Visual Basic, Fortran <NUM>, Perl, COBOL <NUM>, PHP, ABAP, dynamic programming languages such as Python, Ruby, and Groovy, or other programming languages.

Furthermore, the recited order of processing elements or sequences, or the use of numbers, letters, or other designations thereof, are not intended to limit the claimed processes and methods to any order except as may be specified in the claims. Although the above disclosure discusses through various examples what is currently considered to be a variety of useful embodiments of the disclosure, it is to be understood that such detail is solely for that purpose and that the appended claims are not limited to the disclosed embodiments. For example, although the implementation of various components described above may be embodied in a hardware device, it may also be implemented as a software only solution, e.g., an installation on an existing server or mobile device.

Claim 1:
A method for detecting a position and a lane line category of a lane line, implemented on a computing device including at least one processor and at least one storage medium, the method comprising:
obtaining (<NUM>) an image; determining (<NUM>), for each of a plurality of pixels in the image, a semantic category using a trained semantic segmentation network;
determining (<NUM>) one or more pixel sets based on the plurality of pixels according to a predetermined rule, wherein each of the one or more pixel sets includes one or more pixels of a same semantic category;
performing a binarization operation (<NUM>) to obtain binary information of each of the plurality of pixels, wherein the binary information indicates edge pixels; and
for each of the one or more pixel sets,
performing a fitting operation (<NUM>) on the one or more pixels in the pixel set to obtain a fitting line corresponding to a lane line;
determining (<NUM>) the position of the lane line based on the fitting line; and
determining, based on at least the fitting line and the binary information of the one or more pixels in the pixel set, the lane line category of the lane line, wherein the lane line category of the lane line indicates whether the lane line is a dashed line or a solid line, including:
determining (<NUM>), for each pixel in the one or more pixel sets, based on a fitted coordinate of said pixel and a lane line width, a searching range for the pixel, wherein the fitted coordinate is determined based on a line equation, which is obtained by the fitting operation, and based on an actual coordinate of the pixel;
determining (<NUM>) a reference number count of reference pixels by traversing (<NUM>) each pixel of the one or more pixel sets;
in response to a determination that there are no edge pixels within the searching range of the traversed pixel, designating the pixel as a reference pixel; and
determining a count of the reference pixels in the one or more pixel sets as the reference number count; and
determining (<NUM>) the lane line category of the lane line by comparing (<NUM>; <NUM>) the reference number count of pixels and a count threshold.