Detecting road condition changes from probe data

Systems, methods, and apparatuses are disclosed for identifying anomalies or changes in road conditions on a roadway location. An initial low rank data matrix of initial vehicle probe data at a plurality of different times for a roadway location is provided, where the initial low rank data matrix represents a baseline of road conditions for the roadway location. A plurality of additional vehicle probe data from at least one vehicle at the roadway location is received. The additional vehicle probe data is added to the initial vehicle probe data of the initial low rank data matrix. The updated data matrix with the compiled probe data is decomposed into a low rank data matrix and a sparse data matrix. A change at the roadway location is identified based on the probe data in the sparse data matrix.

FIELD

The following disclosure relates to detecting road conditions or changes in road conditions using machine learning algorithms.

BACKGROUND

Traffic reporters may rely on traffic information made available by government agencies. Also, online traffic reporting resources may suffer from infrequent updates, data entry errors, or delayed data input. These factors cause the traffic reporter to fail to timely report a major traffic incident or congestion, or continue to report an incident or congestion well after the incident or congestion has been cleared. Therefore, providing real time, accurate traffic information, such as for use in a navigation system (and, in particular, for an autonomous vehicle), is a continuing effort.

To enable automated driving, route validation and planning beyond a vehicle's sensor range is important. For example, it is important to accurately and timely detect low latency road condition changes and abnormalities such as lane closures and slipperiness of the road. Detection of road condition changes or abnormalities is also important for lane positioning as the road structure and identified objects may function as references for matching vehicle sensor perception. Near real-time updates of the road structure and objects may provide correct lane positioning for the automated vehicle navigation system.

SUMMARY

Systems, methods, and apparatuses are provided for detecting road conditions or changes in road conditions at a roadway location. In one embodiment, the method comprises providing an initial low rank data matrix of initial vehicle probe data at a plurality of different times for a roadway location, the initial low rank data matrix representing a baseline of road conditions for the roadway location. The method further comprises receiving, using a processor, a plurality of additional vehicle probe data from at least one vehicle at the roadway location. The method further comprises adding the additional vehicle probe data to the initial vehicle probe data of the initial low rank data matrix to provide an updated data matrix comprising compiled probe data. The method further comprises decomposing the compiled probe data in the updated data matrix into a low rank data matrix and a sparse data matrix. The method further comprises identifying a change at the roadway location based on probe data in the sparse data matrix.

In one embodiment, the apparatus comprises at least one processor and at least one memory including computer program code for one or more programs; the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least perform: (1) provide an initial low rank data matrix of initial vehicle probe data at a plurality of different times for a roadway location, the initial low rank data matrix representing a baseline of road conditions for the roadway location; (2) receive a plurality of additional vehicle probe data from at least one vehicle at the roadway location; (3) add the additional vehicle probe data to the initial vehicle probe data of the initial low rank data matrix to provide an updated data matrix comprising compiled probe data; (4) decompose the compiled probe data in the updated data matrix into a low rank data matrix and a sparse data matrix; and (5) identify a change at the roadway location based on probe data in the sparse data matrix.

DETAILED DESCRIPTION

The following embodiments include the identification of anomalies or changes in road conditions on a roadway location. The systems, methods, and apparatuses include collecting or providing an initial low rank data matrix of initial vehicle probe data at a plurality of different times for a roadway location, receiving a plurality of additional vehicle probe data from at least one vehicle at the roadway location is received, adding the additional probe data to provide an updated data matrix, and determining whether there is a change in the road condition based on an analysis of the updated data matrix.

As used herein, a “road” or “roadway” may refer to any traveling lane(s) or pathway(s) that may be capable of being observed by vehicle sensors for road conditions (e.g., markings, width, signs, slipperiness, etc.), or may become capable of being observed by vehicle sensors in the future. Non-limiting examples of a road or roadway includes a highway, city street, bus route, train route, walking/biking pathway, or waterway.

Collecting Initial Probe Data

In order to identify changes in traffic conditions for a roadway location, a baseline of normal or typical traffic conditions for the roadway location is established over a period of time (or up until a defined time t). The baseline may be established for a number of variables or vehicle probe features (discussed in greater detail below). As discussed in greater detail below, the baseline data may be compiled in an initial low rank data matrix.

In certain embodiments, vehicle probe data is collected with a navigation device transported in or on a probe vehicle (e.g., car, truck, motorcycle, bicycle, bus) or on a traveler. The navigation device is configured to calculate probe data such as geographic location, speed, and heading of the probe vehicle or traveler. The probe data may be determined through Global Positioning System (GPS) or another technique. The probe data may include speed, heading, location, timestamp, etc., as obtained from a single sensor such as GPS or a combination of sensors such as GPS, accelerometer, and gyrometer. In certain embodiments, the navigation device generates a message that provides at least one of (1) geographic location, (2) speed, (3) heading, and (4) vehicle identification (including timestamp), and sends the message to a server for processing.

In some embodiments, the vehicle containing the navigation device is an autonomous vehicle or a highly automated driving (HAD) vehicle. As described herein, an “autonomous vehicle” may refer to a self-driving or driverless mode in which no passengers are required to be on board to operate the vehicle. An autonomous vehicle may be referred to as a robot vehicle or an automated vehicle. The autonomous vehicle may include passengers, but no driver is necessary. These autonomous vehicles may park themselves or move cargo between locations without a human operator. Autonomous vehicles may include multiple modes and transition between the modes.

As described herein, a “highly automated driving (HAD) vehicle” may refer to a vehicle that does not completely replace the human operator. Instead, in a highly automated driving mode, the vehicle may perform some driving functions and the human operator may perform some driving functions. Vehicles may also be driven in a manual mode in which the human operator exercises a degree of control over the movement of the vehicle. The vehicles may also include a completely driverless mode. Other levels of automation are possible.

The autonomous or highly automated driving vehicle may include vehicle sensors for identifying the surroundings and the location of the car. The sensors may include GPS, light detection and ranging (LIDAR), radar, and cameras for computer vision. Proximity sensors may aid in parking the vehicle. The proximity sensors may detect the curb or adjacent vehicles. The autonomous or highly automated driving vehicle may optically track and follow lane markings or guide markings on the road.

FIG. 1illustrates example sensors for a vehicle124. For example, an engine sensor111may include throttle sensor that measures a position of a throttle of the engine or a position of an accelerator pedal, a brake senor that measures a position of a braking mechanism or a brake pedal, or a speed sensor that measures a speed of the engine or a speed of the vehicle wheels. A vehicle sensor113may include a steering wheel angle sensor, a speedometer sensor, or a tachometer sensor.

An additional vehicle sensor115may be a camera, a light detection and ranging (LIDAR) sensor, a radar sensor, or an ultrasonic sensor. The vehicle sensor115may determine road status such as the shape or turns of the road, the existence of speed bumps, the existence of pot holes, the wetness of the road, or the existence or ice, snow, or slush.

In certain embodiments, the vehicle sensor115may identify lane or guide markings on the road, such as the left and right boundaries of the lane the vehicle is traveling (i.e., the current lane), as well as any immediate adjacent lane boundaries (i.e., the adjacent left lane boundary and the adjacent right lane boundary), to the extent the adjacent lanes exist. This information may be used (in conjunction with lane marking identifications from other vehicles) to determine the number of lanes of travel on the roadway at the roadway location. In certain embodiments, based on a determined number of lanes, the vehicle sensor115may assist in identifying which lane (i.e., a lane number) the vehicle is traveling.

The vehicle sensor115may also identify the angle between the trajectory path of the vehicle and a lane marking. The vehicle sensor115may also be able to gather information to determine the vehicle's distance from the current lane's left or right boundary.

Reporting Probe Data

The vehicle or navigation device may communicate with a network, wherein the probe data collected from the navigation device or vehicle sensors may be transmitted through the network and stored in a database or at the server. The transmitted probe data may then be collected and analyzed by a processor within the map developer system.

In certain embodiments, the probe data is analyzed to develop a baseline or initial low rank data matrix for analysis with future collected probe data. In establishing the baseline, the vehicle probe data is compiled and saved in an initial low rank data matrix comprising probe data points or feature vectors stored for a plurality of different observations or times up until time t. For example, the initial low rank data matrix may include probe data on a vehicle's speed (s), heading (h), and lane count (c). The data matrix may store this probe data for a number of different observations or times (e.g., times 1, 2, 3, . . . N) at the roadway location. An embodiment for the compiled baseline low rank data matrix is provided below:

The baseline of vehicle probe data may be provided as an initial low rank data matrix for analysis purposes with future collected data

Receiving Additional Probe Data and Forming New Data Matrix

Following development of a baseline data matrix up until time t, the map developer system may receive additional vehicle probe data after time t. The probe data may be collected and transmitted through the network in a similar manner as the initial probe data. In certain embodiments, the additional probe data is collected from a single vehicle at the roadway location at a single time. In other embodiments, the additional probe data is collected from a single vehicle at the roadway location at a plurality of different times. In yet other embodiments, the additional probe data is collected from a plurality of vehicles at the roadway location at a single time. In yet further embodiments, the additional probe data is collected from a plurality of vehicles at the roadway location at a plurality of different times.

In certain embodiments, the additional vehicle probe data is added to the initial vehicle probe data of the initial low rank data matrix to provide an updated data matrix with the compiled probe data (e.g., the baseline probe data and the additional vehicle probe data collected and reported).

Decomposing Data Matrix

In certain embodiments, the updated data matrix containing the compiled probe data may be decomposed into a new low rank data matrix and a sparse data matrix (e.g., abnormal or corrupted measurements). A machine learning algorithm may be used to decompose the compiled probe data. In certain embodiments, the machine learning algorithm is a principal component analysis (PCA) algorithm or a robust principal component analysis (RPCA) algorithm.

In such algorithms, vehicle probe data is converted into a set of linearly uncorrelated variables or principal components through an orthogonal transformation, therein providing feature vectors of a covariance matrix.

Through the machine learning algorithm, the feature vectors of the compiled probe data are analyzed for trends and abnormalities (e.g., corrupted measurements). The rank of the feature vectors may be determined within the updated data matrix, wherein the rank represents the largest number of independent vectors in the matrix. Through the decomposition of the matrix, the probe data having feature vectors that are similar to other probe data are decomposed into the low rank data matrix, while abnormal data are sorted into the sparse data matrix.

FIG. 2depicts one embodiment of a data matrix decomposed into a low rank data matrix and a sparse data matrix comprising feature vectors, where each feature is provided in different columns at different times (rows). Cumulatively, the two matrices comprise an overall data matrix of all the observations. With regard to the low rank data matrix, the vertical stripes in each column depict each feature vector at different times for the roadway location, with the stripes representing the similarity in each feature vector. With regard to the sparse data matrix, the bright spots indicate the irregular feature at the roadway location at a particular time. For example, identifying the bright spot in a sparse matrix may provide information as to what feature (column #) was different from regular, baseline events at a particular time (row #).

For example, at the roadway location, the vehicle is traveling at a similar speed or heading, or the vehicle is observing a similar number of lanes of travel, as the other probe data in the data matrix. This may be referred to as a regular event or low-rank data.

Where the rank is high, one or more of the feature vectors are considered to be different (e.g., grossly different), and the probe data is sorted or decomposed into the sparse data matrix. In other words, the additional probe data collected and reported contains at least one feature that is different from the baseline probe data feature. For example, the vehicle may be traveling at a different speed or a different heading, or may be observing a different number of lanes of travel in comparison to the compiled probe data in the data matrix. So, if the lane count observation at the roadway location is abnormal from the majority of data in the data matrix, it may mean that there is a lane closure or a road change. If the speed is different, it may mean that there is a traffic jam or incident. If the heading or direction of travel is irregular, it may be the result of construction at the roadway location, where the vehicles have been rerouted to the opposite side of the road.

The determination of whether the probe data in the combined data matrix is decomposed into the low rank data matrix or sparse data matrix may be adjustable based on the algorithm parameters. In other words, the sensitivity on determining whether the probe data is similar or dissimilar from the compiled probe data may be adjustable. For example, the threshold for determining whether the probe data is similar or dissimilar may be a 10%, 20%, or 30% change in the average value or feature vector. In other embodiments, the threshold may be based on the difference of 1 standard deviation, 2 standard deviations, etc.

FIG. 3depicts two examples of varying levels of sensitivity.FIG. 3(a)depicts an example where the sensitivity threshold is high, and a larger number of vehicle probe data observations are considered dissimilar or abnormal from the baseline probe data.FIG. 3(b)depicts an example where the sensitivity threshold is low, and only drastically dissimilar observations are stored in the sparse data matrix. InFIG. 3(b), as pointed out with an arrow, one vehicle reported an abnormal speed at the roadway location at a particular time.

Determining Changes and Reporting Road Conditions

Changes in road conditions may be determined based on the decomposition of the data matrix into the low rank and sparse data matrices. In particular, a change in the road condition at a certain time may be determined based on the probe data populated in the sparse data matrix. In certain embodiments, a change in a road condition at the roadway condition (e.g., speed, heading, or lane count) may be determined when there are a minimum number of abnormal observations populated in the sparse data matrix in a period of time. For example, a change in the road conditions may exist when there are at least 5, 10, 15, 20, or 25 abnormal or high-rank observations populated in the sparse data matrix in a specific period of time (e.g., 5 minutes, 10 minutes, 15 minutes, 30 minutes, 1 hour). For example, the low rank data matrix has a lane count of 4 total lanes, and there are 10 new observations within a 15 minute time frame reporting a lane count of 3 lanes, this may trigger an alert that there is a lane closure at the roadway location.

In another embodiment, a change in a road condition at the roadway condition (e.g., speed, heading, or lane count) may be determined when are minimum percentage of observations in a period of time are abnormal. For example, a change in the road conditions may exist when there at least 50%, 60%, 70%, 80%, or 90% of the observations are considered abnormal or high-rank observations in a specific period of time (e.g., 5 minutes, 10 minutes, 15 minutes, 30 minutes, 1 hour). For example, the low rank data matrix has a vehicle heading direction of West to East, and there are 10 new observations within a 15 minute time frame, and 50% of those observations are abnormal (e.g., a heading in the opposite direction from East to West), this may trigger an alert that there is a re-routing of traffic in an identified lane at the roadway location.

Following determination of a change in the road condition at the roadway location, the change may be published in the map database or reported to a traffic provider or map developer. In other embodiments, the road changes may be reported to an end-user (e.g., a navigation device in a vehicle) over a connected network. The updates to the map database and reports to the traffic provider, map developer, or end-user may take place in real-time, wherein the probe data is reported by the navigation devices in real-time, and following a collection of a threshold level of probe data anomalies, the determined road change is reported.

In some embodiments, in cases of autonomous or HAD vehicles, the vehicle may process the reported data and make a decision on whether to alert the operator or take action. In certain embodiments, the navigation device in the vehicle or another computer system in communication with the navigation device may include instructions for routing the vehicle or generate driving commands for steering the vehicle, shifting gears, increasing and decreasing the throttle, and braking based on the reported data.

Updating Low Rank Data Matrix

In certain embodiments, following reporting of a change in the road condition (e.g., a lane closure), or following continued determinations of anomalies in the road condition, the change in the road condition may be considered “permanent.” As such, future determinations of road anomalies should be compared with a new, updated low rank data matrix (i.e., a new baseline).

Similar to the reporting of a change in the road condition, the determination of whether the road condition is “permanent” may be based on whether there are a minimum number of abnormal observations in a period of time (e.g., when there are at least 5, 10, 15, 20, or 25 abnormal or high-rank observations in a specific period of time, such as 5 minutes, 10 minutes, 15 minutes, 30 minutes, 1 hour). In certain embodiments, the minimum observation threshold may be equal to or higher than the threshold for reporting a road condition change.

In another embodiment, the determination of whether the road condition is “permanent” may be based on when a threshold minimum percentage of observations in a period of time are abnormal (e.g., when there at least 50%, 60%, 70%, 80%, or 90% of the observations are considered abnormal or high-rank observations in a specific period of time such as 5 minutes, 10 minutes, 15 minutes, 30 minutes, 1 hour). In certain embodiments, the minimum percentage threshold may be equal to or higher than the threshold for reporting a road condition change.

In certain embodiments, following a determination that the road condition is permanent, the low rank data may be updated to represent the change. This update may comprise decomposing at least a portion of the current data matrix into a new low rank data matrix and a new sparse matrix, wherein the new low rank data matrix comprises the probe data representing the road condition change. In other embodiments, the low rank data matrix may be updated to represent the change by selecting all of the probe data in the data matrix from a recent period of time (e.g., within the last 5 minutes, 10 minutes, 15 minutes, 30 minutes, 1 hour, 1 day, etc.) and decomposing that subset of probe data to provide the new baseline/low rank data matrix.

Flowchart Embodiment

FIG. 4illustrates one embodiment of a flowchart for predicting a speed limit value and a confidence level of the predicted value. The process of the flowchart may be performed by the camera sensor, navigation device, and processor and/or a server and its processor. Alternatively, another device may be configured to perform one or more of the following acts. Additional, fewer, or different acts may be included.

At act S101, an initial low rank data matrix of initial vehicle probe data for a particular roadway location is provided or collected in a map database. The initial low rank data matrix comprises a plurality of different feature vectors at a plurality of different times for the roadway location. The data matrix may represent a substantially uniform set of probe data, therein providing a baseline for comparison against newly collected, (e.g., real-time) vehicle probe data reported to the map database for analysis.

At act S103, additional vehicle probe data at the roadway location is collected by one or more vehicles and reported to the map database server. At act S105, the additional probe data is added to the initial low rank data matrix to provide an updated data matrix comprising the compiled probe data.

At act S107, the updated data matrix is decomposed into a low rank data matrix and a sparse data matrix. In certain embodiments, a machine learning algorithm such as a robust principal component analysis (RPCA) algorithm is used for the decomposition process.

At act S109, a road condition is identified from the probe data in the sparse data matrix. In certain embodiments, this determination is based on a threshold number of abnormal observations in the sparse data matrix, such as a minimum number of abnormal observations over a period of time or a minimum percentage of abnormal observations in the period of time. Following identification of a road condition, the road condition may be used to update a map database, or the road condition may be reported to a traffic developer or end-user.

At act S111, a road condition may be determined to be “permanent,” wherein the low rank data matrix is updated with the new road condition, therein creating a new baseline for future analysis (wherein the process can continue in perpetuity).

Navigation and Network System

FIG. 5illustrates an example system120for reporting and processing probe data from a navigation device122or vehicle sensor111,113,115associated with a vehicle124or traveler. The system120includes a developer system121, one or more navigation devices122, vehicle sensors, a workstation128, and a network127. Additional, different, or fewer components may be provided. For example, many navigation devices122and/or workstations128connect with the network127. The developer system121includes a server125and a database123. The developer system121may include computer systems and networks of a system operator.

The navigation device122or vehicle sensor may be carried by or installed within a vehicle124. Vehicle sensors, as described withFIG. 1, may include an engine sensor111such as a throttle sensor that measures a position of a throttle of the engine or a position of an accelerator pedal, a brake sensor that measures a position of a braking mechanism or a brake pedal, or a speed sensor that measures a speed of the engine or a speed of the vehicle wheels. A vehicle sensor113may include a steering wheel angle sensor, a speedometer sensor, or a tachometer sensor. An additional vehicle sensor115may be a camera, a light detection and ranging (LIDAR) sensor, a radar sensor, or an ultrasonic sensor. The vehicle sensor115may determine road status such as the shape or turns of the road, the existence of speed bumps, the existence of pot holes, the wetness of the road, or the existence or ice, snow, or slush.

The navigation device122may be a specialized autonomous driving computer. The navigation device122may calculate a vehicle confidence level based on at least one confidence factor. The confidence factors may be based on sensor data collected at the vehicle, environmental data received through the network127, or responsiveness of the vehicle124. Alternatively, the navigation device122may report sensor data to the server125, which calculates the vehicle confidence level.

The navigation device122may be a personal navigation device (“PND”), a portable navigation device smart phone, a mobile phone, a personal digital assistant (“PDA”), a tablet computer, a notebook computer, and/or any other known or later developed mobile device or personal computer. Non-limiting embodiments of navigation devices may also include RDS devices, HD radio devices, mobile phone devices, or car navigation devices such as Garmin or TomTom.

The developer system121includes a server125and a server database123. The developer system121may include computer systems and networks of a system operator such as HERE, NAVTEQ, or Nokia Corporation. The server database123is configured to store the vehicle probe data in a data matrix.

The developer system121, the workstation128, and the navigation device122are coupled with the network127. The phrase “coupled with” is defined to mean directly connected to or indirectly connected through one or more intermediate components. Such intermediate components may include hardware and/or software-based components.

The workstation128may be a general purpose computer including programming specialized for providing input to the server125. For example, the workstation128may provide settings for the server125. The settings may include a value for the predetermined interval that the server125requests the navigation device122to relay current geographic locations. The workstation128may be used to enter data indicative of GPS accuracy to the database123. The workstation128may include at least a memory, a processor, and a communication interface.

The computing resources may be divided between the server125and the navigation device122. In some embodiments, the server125performs a majority of the processing for calculating the vehicle confidence value and the comparison with the confidence threshold. In other embodiments, the computing device122or the workstation128performs a majority of the processing. In addition, the processing is divided substantially evenly between the server125and the computing device122or workstation128.

FIG. 6illustrates an exemplary navigation device122of the system ofFIG. 5. The navigation device122includes a processor200, a memory204, an input device203, a communication interface205, position circuitry207, and a display211. Additional, different, or fewer components are possible for the mobile device/personal computer122.

The processor200may be configured to receive data indicative of the location of the navigation device122from the position circuitry207. The positioning circuitry207, which is an example of a positioning system, is configured to determine a geographic position of the navigation device122. The positioning system may also include a receiver and correlation chip to obtain a GPS signal. The positioning circuitry may include an identifier of a model of the positioning circuitry207. The processor200may access the identifier and query a database or a website to retrieve the accuracy of the positioning circuitry207based on the identifier. The positioning circuitry207may include a memory or setting indicative of the accuracy of the positioning circuitry.

FIG. 7illustrates an exemplary server125of the system ofFIG. 5. The server125includes a processor300, a communication interface305, and a memory301. The server125may be coupled to a database123and a workstation128. The workstation128may be used as an input device for the server125. In addition, the communication interface305is an input device for the server125. In certain embodiments, the communication interface305may receive data indicative of use inputs made via the workstation128or the navigation device122.

The navigation device processor200and/or the server processor300may include a general processor, digital signal processor, an application specific integrated circuit (ASIC), field programmable gate array (FPGA), analog circuit, digital circuit, combinations thereof, or other now known or later developed processor. The navigation device processor200and/or the server processor300may be a single device or combinations of devices, such as associated with a network, distributed processing, or cloud computing.

The navigation device processor200and/or the server processor300may also be configured to cause an apparatus to at least perform at least one of methods described above. For example, the navigation device processor200may be configured to collect and transmit vehicle probe data for a roadway location.

In another embodiment, the server processor300may be configured to compile or provide a baseline data matrix of vehicle probe data for the roadway location. The server processor300may also be configured to receive a plurality of additional vehicle probe data from at least one vehicle at the roadway location, and run an algorithm to decompose the combined probe data in a data matrix into the low rank data matrix and sparse data matrix. The server processor300may also be configured to identify a change at the roadway location based on the probe data in the sparse data matrix.

The memory204and/or memory301may be a volatile memory or a non-volatile memory. The memory204and/or memory301may include one or more of a read only memory (ROM), random access memory (RAM), a flash memory, an electronic erasable program read only memory (EEPROM), or other type of memory. The memory204and/or memory301may be removable from the navigation device122, such as a secure digital (SD) memory card.

The communication interface205and/or communication interface305may include any operable connection. An operable connection may be one in which signals, physical communications, and/or logical communications may be sent and/or received. An operable connection may include a physical interface, an electrical interface, and/or a data interface. The communication interface205and/or communication interface305provides for wireless and/or wired communications in any now known or later developed format.