SYSTEMS AND METHODS FOR PREDICTING A VEHICLE TRAJECTORY

Embodiments of the disclosure provide methods and systems for predicting a trajectory of a vehicle. An exemplary system includes a communication interface configured to receive a map of an area in which the vehicle is traveling and sensor data acquired associated with the vehicle. The system includes at least one processor configured to position the vehicle in the map and identify one or more objects surrounding the vehicle based on the positioning of the vehicle. The at least one processor is further configured to extract features of the vehicle and the one or more objects from the sensor data. The at least one processor is also configured to determine a plurality of candidate trajectories, determine a probability for each candidate trajectory based on the extracted features, and identify the candidate trajectory with the highest probability as the predicted trajectory of the vehicle.

TECHNICAL FIELD

The present disclosure relates to systems and methods for predicting a vehicle trajectory, and more particularly, to systems and methods for predicting a vehicle trajectory using features extracted from map and sensor data.

BACKGROUND

Vehicles share roads with other vehicles, bicycles, pedestrians, and objects, such as traffic signs, road blocks, fences, etc. Therefore, drivers need to constantly adjust driving to avoid colliding the vehicle with such obstacles. While some obstacles are generally static and therefore easy to avoid, some others might be moving. For a moving obstacle, the driver has to not only observe its current position but to predict its moving trajectory in order to determine its future positions. For example, another vehicle on the road coming towards the vehicle may go straight, stop, or make turns. The driver typically makes the prediction based on observations such as turn signals provided by the coming vehicle, the vehicle's traveling speed, etc.

Automatous driving vehicles need to make similar decisions to avoid obstacles. Therefore, automatous driving technology relies heavily on automated prediction of other vehicles' trajectories. However, existing prediction systems and methods are limited by the vehicle's ability to “see” (e.g., to collect relevant data), ability to process the data, and ability to make accurate predictions based on the data. Accordingly, automatous driving vehicles can benefit from improvements to the existing prediction systems and methods.

Embodiments of the disclosure improve the existing prediction systems and methods in automatous driving by providing systems and methods for predicting a vehicle trajectory using features extracted from map and sensor data.

SUMMARY

Embodiments of the disclosure provide a system for predicting a trajectory of a vehicle. The system includes a communication interface configured to receive a map of an area in which the vehicle is traveling and sensor data acquired associated with the vehicle. The system includes at least one processor configured to position the vehicle in the map and identify one or more objects surrounding the vehicle based on the positioning of the vehicle. The at least one processor is further configured to extract features of the vehicle and the one or more objects from the sensor data. The at least one processor is also configured to determine a plurality of candidate trajectories, determine a probability for each candidate trajectory based on the extracted features, and identify the candidate trajectory with the highest probability as the predicted trajectory of the vehicle.

Embodiments of the disclosure also provide a method for predicting a trajectory of a vehicle. The method includes receiving, by a communication interface, a map of an area in which the vehicle is traveling and sensor data acquired associated with the vehicle. The method further includes positioning, by at least one processor, the vehicle in the map and identifying, by the at least one processor, one or more objects surrounding the vehicle based on the positioning of the vehicle. The method also includes extracting, by the at least one processor, features of the vehicle and the one or more objects from the sensor data. The method additionally includes determining, by the at least one processor, a plurality of candidate trajectories, determining, by the at least one processor, a probability for each candidate trajectory based on the extracted features, and identifying the candidate trajectory with the highest probability as the predicted trajectory of the vehicle.

Embodiments of the disclosure further provide a non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one processor, causes the at least one processor to perform operations. The operations include receiving a map of an area in which the vehicle is traveling and sensor data acquired associated with the vehicle. The operations further include positioning the vehicle in the map and identifying one or more objects surrounding the vehicle based on the positioning of the vehicle. The operations further include extracting features of the vehicle and the one or more objects from the sensor data. The operations also include determining a plurality of candidate trajectories, determining a probability for each candidate trajectory based on the extracted features, and identifying the candidate trajectory with the highest probability as the predicted trajectory of the vehicle.

DETAILED DESCRIPTION

FIG. 1illustrates a schematic diagram of an exemplary cross-road100and exemplary vehicles (e.g., vehicles120and130) traveling therein, according to embodiments of the disclosure. As shown inFIG. 1, cross-road100includes two roads, one is shown in the vertical direction (referred to as “road A”) and another is shown in the horizontal direction (referred to as “road B”), crossing each other, and traffic lights140at the crossing. For ease of description, road A is illustrated to extend in the North-South direction, and road B is illustrated to extend in the East-West direction. It is contemplated that roads A and B can extend in any other directions, and are not necessarily perpendicular to each other.

Each of road A and road B is shown as a two-way road. For example, road B includes first direction lanes102and104and second direction lanes108and110. The first and second directions may be opposite to each other and separated by a divider106. It is contemplated that one or both of the roads may be one-way and/or have more or less lanes.

Various vehicles may be traveling on the roads in both directions. For example, vehicle120may be traveling east-bound on first direction lane102, and vehicle130may be traveling west-bound on second direction lane103. In some embodiments, vehicles120and130may be electric vehicles, fuel cell vehicles, hybrid vehicles, or conventional internal combustion engine vehicles. In some embodiments, vehicle120may be an autonomous or semi-autonomous vehicle.

The vehicle traffic at cross-road100may be regulated by traffic lights140. Traffic lights140may be installed in one or both directions. In some embodiments, traffic lights140may include lights in three colors: red, yellow and green, to signal the right of way at cross-road100. In some embodiments, traffic lights140may additionally include turn protection lights to regulate the left, right, and/or U-turns at cross-road100. For example, a left turn protection light may allow vehicles in certain lanes (usually the left-most lane) to turn left without having to yield to vehicles traveling straight in the opposite direction.

In some embodiments, vehicle120may be equipped with or in communication with a vehicle trajectory prediction system (e.g., system200shown inFIG. 2) to predict the trajectory of another vehicle on the road, such as vehicle130, in order to make decisions to avoid that vehicle in its own travel path. For example, vehicle130may possibly travel in four candidate trajectories: a candidate trajectory151to make a right-turn, a candidate trajectory152to go straight, a candidate trajectory153to make a left-turn, and a candidate trajectory154to make a U-turn. Consistent with embodiments of the present disclosure, the vehicle trajectory prediction system may make “observations” (e.g., through various sensors) of vehicle130and the surrounding objects, such as traffic light(s)140, traffic signs at cross-road100, and other vehicles on the roads, etc. The vehicle trajectory prediction system then makes a prediction which candidate trajectory vehicle130may likely follow based on these observations. In some embodiments, the prediction may be preformed using a learning model, such as a neural network. In some embodiments, probabilities may be determined for the respective candidate trajectories151-154.

FIG. 2illustrates a schematic diagram of an exemplary system200for predicting a vehicle trajectory, according to embodiments of the disclosure. System200may be used in cross-road100illustrated inFIG. 1or similar settings. For ease of illustration, a simplified cross-road setting is used inFIG. 2. However, it is understood that system200is also applicable in other cross-road settings. System200may include a vehicle trajectory prediction server210(also referred to as server210for simplicity). Server210can be a general-purpose server configured or programmed to predict vehicle trajectories or a proprietary device specially designed for predicting vehicle trajectories. It is contemplated that server210can be a stand-alone server or an integrated component of a stand-alone server. In some embodiments, server210may be integrated into a system onboard a vehicle, such as vehicle120.

As illustrated inFIG. 2, server210may receive and analyze data collected by various sources. For example, data may be continuously, regularly, or intermittently captured by one or more sensors220equipped along a road and/or one or more sensors230equipped on vehicle120driving through lane102. Sensors220and230may include radars, LiDARs, cameras (such as surveillance cameras, monocular/binocular cameras, video cameras), speedometers, or any other suitable sensors to capture data characterizing vehicle130and objects surrounding vehicle130, such as traffic light140. For example, sensors220may include one or more surveillance cameras that capture images of vehicle130and traffic light140.

In some embodiments, sensors230may include a LiDAR that measures a distance between vehicle120and vehicle130, and the position of vehicle130in a 3-D map. In some embodiments sensor230may also include a GPS/IMU (inertial measurement unit) sensor to capture position/pose data of vehicle120. In some embodiments, sensors230may additionally include cameras to capture images of vehicle130and traffic light140. Since the images captured by sensors220and sensors230are from different angles, they may supplement each other to provide more detailed information of vehicle130and surrounding objects. In some embodiments, sensors220and230may acquire data that tracks the trajectories of moving objects, such as vehicles, pedestrians, etc.

In some embodiments, sensors230may be equipped on vehicle120and thus travel with vehicle120. For example,FIG. 3illustrates an exemplary vehicle120with sensors340-360equipped thereon, according to embodiments of the disclosure. Vehicle120may have a body310, which may be any body style, such as a sports vehicle, a coupe, a sedan, a pick-up truck, a station wagon, a sports utility vehicle (SUV), a minivan, or a conversion van. In some embodiments, vehicle120may include a pair of front wheels and a pair of rear wheels320, as illustrated inFIG. 3. However, it is contemplated that vehicle120may have less wheels or equivalent structures that enable vehicle120to move around. Vehicle120may be configured to be all wheel drive (AWD), front wheel drive (FWR), or rear wheel drive (RWD). In some embodiments, vehicle120may be configured to be an autonomous or semi-autonomous vehicle.

As illustrated inFIG. 3, sensors230ofFIG. 2may include various kinds of sensors340,350, and360, according to embodiments of the disclosure. Sensor340may be mounted to body310via a mounting structure330. Mounting structure330may be an electro-mechanical device installed or otherwise attached to body310of vehicle120. In some embodiments, mounting structure330may use screws, adhesives, or another mounting mechanism. Vehicle120may be additionally equipped with sensors350and360inside or outside body310using any suitable mounting mechanisms. It is contemplated that the manners in which sensors340-360can be equipped on vehicle120are not limited by the example shown inFIG. 3and may be modified depending on the types of sensors340-360and/or vehicle120to achieve desirable sensing performance.

Consistent with some embodiments, sensor340may be a LiDAR that measures the distance to a target by illuminating the target with pulsed laser lights and measuring the reflected pulses. Differences in laser return times and wavelengths can then be used to make digital 3-D representations of the target. For example, sensor340may measure the distance between vehicle120and vehicle130or other objects. The light used for LiDAR scan may be ultraviolet, visible, or near infrared. Because a narrow laser beam can map physical features with a very high resolution, a LiDAR scanner is particularly suitable for positioning objects in a 3-D map. For example, a LiDAR scanner may capture point cloud data, which may be used to position vehicle120, vehicle130and/or other objects.

In some embodiments, sensors350may include one or more cameras mounted on body310of vehicle120. AlthoughFIG. 3shows sensors350as being mounted at the front of vehicle120, it is contemplated that sensors350may be mounted or installed at other positions of vehicle120, such as on the sides, behind the mirrors, on the windshields, on the racks, or at the rear end. Sensors350may be configured to capture images of objects surrounding vehicle120, such as other vehicles on the roads (including, e.g., vehicle130), traffic light(s)140, and/or traffic signs. In some embodiments, the cameras may be monocular or binocular cameras. The binocular cameras may acquire data indicating depths of the objects (i.e., the distances of the objects from the cameras). In some embodiments, the cameras may be video cameras that capture image frames over time, thus recording the movements of the objects.

As illustrated inFIG. 3, vehicle120may be additionally equipped with sensor360, which may include sensors used in a navigation unit, such as a GPS receiver and one or more IMU sensors. A GPS is a global navigation satellite system that provides geolocation and time information to a GPS receiver. An IMU is an electronic device that measures and provides a vehicle's specific force, angular rate, and sometimes the magnetic field surrounding the vehicle, using various inertial sensors, such as accelerometers and gyroscopes, sometimes also magnetometers. By combining the GPS receiver and the IMU sensor, sensor360can provide real-time pose information of vehicle120as it travels, including the positions and orientations (e.g., Euler angles) of vehicle120at each time point.

Consistent with the present disclosure, sensors340-360may communicate with server210via a network to transmit the sensor data continuously, or regularly, or intermittently. In some embodiments, any suitable network may be used for the communication, such as a Wireless Local Area Network (WLAN), a Wide Area Network (WAN), wireless communication networks using radio waves, a cellular network, a satellite communication network, and/or a local or short-range wireless network (e.g., Bluetooth™).

Referring back toFIG. 2, althoughFIG. 2only illustrates sensors230equipped on vehicle120, it is contemplated that similar sensors may also be equipped on other vehicles on the roads, including vehicle130. For example, vehicle130may be equipped with a LiDAR, one or more cameras, and/or a GPS/IMU sensor. These sensors may also communicate with server210to provide additional sensor data to aid the prediction.

As shown inFIG. 2, system200may further include a 3-D map database240. 3-D map database240may store 3-D maps. The 3-D maps may include maps that cover different regions and areas. For example, a 3-D map (or map portion) may cover the area of cross-road100. In some embodiments, server210may communicate with 3-D map database240to retrieve a relevant 3-D map (or map portion) based on the position of vehicle120. For example, map data containing the GPS position of vehicle120and its surrounding area may be retrieved. In some embodiments, 3-D map database240may be an internal component of server210. For example, the 3-D maps may be stored in a storage of server210. In some embodiments, 3-D map database240may be external of server210and the communication between 3-D map database240and server210may occur via a network, such as the various kinds of networks described above.

Server210may be configured to analyze the sensor data received from sensors230(e.g., sensors340-360) and the map data received from 3-D map database240to predict the trajectories of other vehicles on the roads, such as vehicle130.FIG. 4is a block diagram of an exemplary server210for predicting a vehicle trajectory, according to embodiments of the disclosure. Server210may include a communication interface402, a processor404, a memory406, and a storage408. In some embodiments, server210may have different modules in a single device, such as an integrated circuit (IC) chip (implemented as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA)), or separate devices with dedicated functions. Components of server210may be in an integrated device, or distributed at different locations but communicate with each other through a network (not shown).

Communication interface402may send data to and receive data from components such as sensors220and230via direct communication links, a Wireless Local Area Network (WLAN), a Wide Area Network (WAN), wireless communication networks using radio waves, a cellular network, and/or a local wireless network (e.g., Bluetooth or WiFi), or other communication methods. In some embodiments, communication interface402can be an integrated services digital network (ISDN) card, cable modem, satellite modem, or a modem to provide a data communication connection. As another example, communication interface402can be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links can also be implemented by communication interface402. In such an implementation, communication interface402can send and receive electrical, electromagnetic or optical signals that carry digital data streams representing various types of information via a network.

Consistent with some embodiments, communication interface402may receive sensor data401acquired by sensors220and/or230, as well as map data403provided by 3-D map database240, and provide the received information to memory406and/or storage408for storage or to processor404for processing. Sensor data401may include information capturing vehicles (such as vehicle130) and other objects surrounding the vehicles. Sensor data401may contain data captured over time that characterize the movements of the objects. In some embodiments, map data403may include point cloud data.

Communication interface402may also receive a learning model405. In some embodiments, learning model405may be applied by processor404to predict vehicle trajectories based on features extracted from sensor data401and map data403. In some embodiments, learning model405may be a predictive model, such as a decision tree learning model. A decision tree uses observations of an item (represented in the branches) to predict a target value of the item (represented in the leaves). In some embodiments, gradient boosting may be combined with the decision tree learning model to form a prediction model as an ensemble of decision trees. For example, learning model405may become a Gradient Boosting Decision Tree model formed with stage-wise decision trees.

In some embodiments, learning model405may be trained using known vehicle trajectories and their respective sample features, such as semantic features including the vehicle speed, the lane markings of vehicle lane, the status of the traffic light, the orientation of the vehicle, vehicle turn signals, vehicle breaking signals, etc. The sample features may additionally include non-semantic features extracted from data descriptive of the vehicle movements. In some embodiments, learning model405may be trained by server210or another computer/server ahead of time.

Processor404may include any appropriate type of general-purpose or special-purpose microprocessor, digital signal processor, or microcontroller. Processor404may be configured as a separate processor module dedicated to predicting vehicle trajectories. Alternatively, processor404may be configured as a shared processor module for performing other functions related to or unrelated to vehicle trajectory predictions. For example, the shared processor may further make autonomous driving decision based on the predicted vehicle trajectories.

As shown inFIG. 4, processor404may include multiple modules, such as a positioning unit440, an object identification unit442, a feature extraction unit444, a trajectory prediction unit446, and the like. These modules (and any corresponding sub-modules or sub-units) can be hardware units (e.g., portions of an integrated circuit) of processor404designed for use with other components or to execute part of a program. The program may be stored on a computer-readable medium (e.g., memory406and/or storage408), and when executed by processor404, it may perform one or more functions. AlthoughFIG. 4shows units440-446all within one processor404, it is contemplated that these units may be distributed among multiple processors located near or remotely with each other.

Positioning unit440may be configured to position the vehicle whose trajectory is being predicted (e.g., vehicle130) in map data403. In some embodiments, sensor data401may contain various data captured of the vehicle to assist the positioning. For example, LiDAR data captured by sensor340mounted on vehicle120may reveal the position of vehicle130in the point cloud data. In some embodiments, the point cloud data captured of vehicle130may be matched with map data401to determine the vehicle's position. In some embodiments, positioning methods such as simultaneous localization and mapping (SLAM) may be used to position the vehicle.

In some embodiments, the positions of the vehicle (e.g., vehicle130) may be labeled on map data401. For example, a subset of point cloud data P1is labeled as corresponding to vehicle130at time T1, a subset of point cloud data P2is labeled as corresponding to vehicle130at time T2, and a subset of point cloud data P3is labeled as corresponding to vehicle130at time T3, etc. The labeled subsets indicate the existing moving trajectory and moving speed of the vehicle.

Object identification unit442may identify objects surrounding the vehicle. These objects may include, e.g., traffic lights104, traffic signs, lane markings, and other vehicles, etc. In some embodiments, various image processing methods, such as image segmentation, classification, and recognition method, may be applied to identify the objects. In some embodiments, machine learning techniques may also be applied for the identification. Such objects may provide additional information useful to the vehicle trajectory prediction. For example, if the vehicles travelling on a right-turn only lane, it is more likely that it is going to turn right than turning left. Alternatively, if the traffic light regulating the lane is red, the vehicle will likely not move immediately. If there is a no U-turn sign at the crossing, the vehicle is unlikely going to make a U-turn.

Feature extraction unit444may be configured to extract features from sensor data401and map data403that are indicative of a future trajectory of a vehicle. The features extracted may be semantical or non-semantical. Semantical features may include, e.g., the vehicle speed, the lane markings of vehicle lane (indicating a travel restriction of the lane), the status of the traffic light (including the type of light that is on and the color of the light), the vehicle heading direction, vehicle turn signals, vehicle braking signals, etc. Various feature extraction tools may be used, such as image segmentation, object detection, etc. For example, lane markings (e.g., left-turn only arrow, right-turn only arrow, go-straight only arrow, or combination arrows) can be detected from the sensor data based on color and/or contrast information as the markings are usually in white paint and road surface is usually black or gray in color. When color information is available, lane markings can be identified based on their distinct color (e.g., white). When grayscale information is available, lane markings can be identified based on their different shading (e.g., lighter gray) in contrast to the background (e.g., darker gray for regular road pavements). As another example, traffic light signals, vehicle turn signals, and braking signals can be detected by detecting the change (e.g., resulting from blinking, flashing, or color changing) in image pixel intensities. In some embodiments, machine learning techniques may also be applied to extract the feature(s).

Trajectory prediction unit446may predict the vehicle trajectory using the extracted features. In some embodiments, trajectory prediction unit446may determine a plurality of candidate trajectories, such as candidate trajectories151-154for vehicle130(shown inFIG. 1). In some embodiments, trajectory prediction unit446may apply learning model405for the prediction. For example, learning model405may determine a probability for each candidate trajectory based on the extracted features. Alternatively, learning model405may rank the candidate trajectories by assigning ranking numbers to them. In some embodiments, the candidate trajectory with the highest probability or ranking may be identified as the predicted trajectory of the vehicle.

In some embodiments, before applying learning model405, trajectory prediction unit446may first remove one or more candidate trajectories that conflicts with any of the features. For example, if the vehicle is on a lane with right-turn only lane marking, and the vehicle is signaling a right-turn, a left-turn trajectory and a U-turn trajectory may be eliminated since the probably that the vehicle will turn left or make a U-turn under such conditions is substantially low. As another example, if the vehicle is on the left-most lane and signaling a left-turn, but a traffic sign forbids a U-turn, the U-turn trajectory may be eliminated. By removing certain candidate trajectories, trajectory prediction unit446simplifies the prediction task and conserves processing power of processor404.

In some embodiments, trajectory prediction unit446may compare the determined probabilities for the respective candidate trajectories with a threshold. If none of the candidate trajectory has a probability exceeding the threshold, trajectory prediction unit446may determine that the prediction is not sufficiently reliable and additional “observations” are necessary to improve the prediction. In some embodiments, trajectory prediction unit446may determine what additional sensor data can be acquired and generate control signals to be transmitted to sensors220and/or230for capturing the additional data. For example, it may be determined that the LiDAR should be tilted at a different angle or that the camera should adjust its focal point. The control signal may be provided to sensors220and/or230via communication interface402.

Memory406and storage408may include any appropriate type of mass storage provided to store any type of information that processor404may need to operate. Memory406and storage408may be a volatile or non-volatile, magnetic, semiconductor, tape, optical, removable, non-removable, or other type of storage device or tangible (i.e., non-transitory) computer-readable medium including, but not limited to, a ROM, a flash memory, a dynamic RAM, and a static RAM. Memory406and/or storage408may be configured to store one or more computer programs that may be executed by processor404to perform vehicle trajectory functions disclosed herein. For example, memory406and/or storage408may be configured to store program(s) that may be executed by processor404to predict the vehicle trajectory based on features extracted from the sensor data401captured by various sensors220and/or230, and map data403.

Memory406and/or storage408may be further configured to store information and data used by processor404. For instance, memory406and/or storage408may be configured to store sensor data401captured by sensors220and/or230, map data403received from 3-D map database240, and learning model405. Memory406and/or storage408may also be configured to store intermediate data generated by processor404during feature extraction and trajectory prediction, such as the features, the candidate trajectories, and the calculated probabilities for the candidate trajectories. The various types of data may be stored permanently, removed periodically, or disregarded immediately after each frame of data is processed.

FIG. 5illustrates a flowchart of an exemplary method500for predicting a vehicle trajectory, according to embodiments of the disclosure. For example, method500may be implemented by system200that includes, among other things, server210and sensors220and230. However, method500is not limited to that exemplary embodiment. Method500may include steps S502-S518as described below. It is to be appreciated that some of the steps may be optional to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown inFIG. 5. For description purpose, method500will be described as predicting the trajectory of vehicle130(as shown inFIG. 1) to aid autonomous driving decisions of vehicle120(as shown inFIG. 1). Method500, however, can be implemented for other applications that can benefit from accurate predictions of vehicle trajectories.

In step S502, server210receives a map of the area vehicle130is traveling. In some embodiments, server210may determine the position of vehicle120based on, e.g., the GPS data collected by sensor360, and identify a map area surrounding the position. If vehicle130is also connected with server210via a network, server210may alternatively identify the map area surrounding the GPS position of vehicle130. Server210may receive the relevant 3-D map data, e.g., map data403, from 3-D map database240.

In step S504, server210receives the sensor data capturing vehicle130and surrounding objects. In some embodiments, the sensor data may be captured by various sensors such as sensors220installed along the roads and/or sensors230(including, e.g., sensors340-360) equipped on vehicle120. The sensor data may include vehicle speed acquired by a speedometer, images (including video images) acquired by cameras, point cloud data acquired by a LiDAR, etc. In some embodiments, the sensor data may be captured over time to track the movement of vehicle130and surrounding objects. The sensors may communicate with server210via a network to transmit the sensor data, e.g., sensor data401, continuously, or regularly, or intermittently.

Method500proceeds to step S506, where server210positions vehicle130in the map. In some embodiments, the point cloud data captured of vehicle130, e.g., by sensor340, may be matched with map data403to determine the vehicle's position in the map. In some embodiments, positioning methods such as SLAM may be used to position vehicle130. In some embodiments, the positions of vehicle130at different time points may be labeled on map data403to trace the prior trajectory and moving speed of the vehicle. Labeling of the point cloud data may be performed by server210automatically or with human assistance.

In step S508, server210identifies other objects surrounding vehicle130. Features of such objects may provide additional information useful for predicting the trajectory of vehicle130. For example, these objects may include, e.g., traffic lights104, traffic signs, lane markings, and other vehicles at cross-road100, etc. In some embodiments, various image processing methods and machine learning methods may be implemented to identify the objects.

In step S510, server210extracts features of vehicle130and its surrounding objects from sensor data401and map data403. In some embodiments, the features extracted may include semantical or non-semantical that are indicative of future trajectory of the vehicle. For example, extracted features of vehicle130may include, e.g., the vehicle speed, the vehicle heading direction, vehicle turn signals, vehicle braking signals, etc. Extracted features of surrounding objects may include, e.g., the lane markings of vehicle lane (indicating a travel restriction of the lane), the status of the traffic light (including the type of light that is on and the color of the light), and information on the traffic signs. In some embodiments, various feature extraction methods including image processing methods and machine learning methods may be implemented.

In step S512, server210determines multiple candidate trajectories for vehicle130. Candidate trajectories are possible trajectories vehicle130may follow. For example, vehicle130may follow one of the four candidate trajectories151-154(shown inFIG. 1), i.e., to turn right, go straight, turn left, or make a U-turn at cross-road100. In some embodiments, server210may remove one or more candidate trajectories that conflicts with any of the features. This optional filtering step may help simplify the prediction task and conserve processing power of server210. For example, if the vehicle is on a lane with right-turn only lane marking, and the vehicle is signaling a right-turn, a left-turn trajectory and a U-turn trajectory may be eliminated since the probability that the vehicle will turn left or make a U-turn under such conditions is substantially low.

Method500proceeds to step S514to determine a probability for each candidate trajectory. In some embodiments, server210may apply learning model405for the prediction. In some embodiments, learning model405may be a predictive model, such as a decision tree learning model. For example, learning model405may be a Gradient Boosting Decision Tree model. In some embodiments, learning model405may be trained using known vehicle trajectories and their respective sample features. In step S514, learning model405may be applied to determine a probability for each candidate trajectory based on the extracted features. For example, it may be determined that vehicle130has a 10% probability to follow candidate trajectory151to make a right-turn, 50% probability to follow candidate trajectory152to go straight, 30% probability to follow candidate trajectory153to make a left-turn, and 10% probability to follow candidate trajectory154to make a U-turn.

In step S516, server210may compare the probabilities with a predetermined threshold. In some embodiments, the predetermined threshold may be a percentage higher than 50%, such as 60%, 70%, 80%, or 90%. If no probability is higher than the threshold (S516: No), the prediction may be considered unreliable. In some embodiments, method500may return to step S504to receive additional sensor data to improve the prediction. In some embodiments, server210may determine what additional sensor data can be acquired and generate control signals to direct sensors220and/or230to capture the additional data to be received in step S504.

If at least the highest probability is higher than the threshold (S516: Yes), server210may predict the vehicle trajectory in step S518by selecting the corresponding candidate trajectory from the candidate trajectories. In some embodiments, the candidate trajectory with the highest probability may be identified as the predicted trajectory of the vehicle. For example, candidate trajectory152may be selected as the predicted trajectory of vehicle130when it has the highest probability.

The prediction result provided by method500may be used to aid vehicle controls or driver's driving decisions. For example, an autonomous vehicle may make automated control decisions based on the predicted trajectories of other moving vehicles in order not to collide with them. The prediction may also be used to help alerting a driver to adjust his intended driving path and/or speed to avoid collision. For example, audio alerts such as beeping may be provided.

It will be apparent to those skilled in the art that various modifications and variations can be made to the disclosed system and related methods. Other embodiments will be apparent to those skilled in the art from consideration of the specification and practice of the disclosed system and related methods.

It is intended that the specification and examples be considered as exemplary only, with a true scope being indicated by the following claims and their equivalents.