Patent Description:
This specification describes technologies relating to systems and methods for monitoring and reporting traffic information.

In general, one innovative aspect of the subject matter described in this specification can be embodied in a method as provided in claim <NUM>.

In general, another innovative aspect of the subject matter described in this specification can be embodied in a non-transitory computer storage medium encoded with a computer program, according to claim <NUM>.

These and other embodiments can each optionally include one or more of the following features. Methods can include detecting a plate number of the second vehicle; and transmitting, to the application server, the plate number of the second vehicle.

Methods can include obtaining proximity data of a set of surrounding vehicles that are within a defined area around the first vehicle; determining a speed of one or more of the surrounding vehicles from the set; determining that a traffic delay is occurring based on the determined speed of the one or more of the surrounding vehicles; in response to determining that the traffic delay is occurring: generating traffic delay data; and transmitting the traffic delay data to the application server.

Generating traffic delay data can include generating traffic delay data that specifies one or more of speed information for the set of surrounding vehicles and a plate number of the second vehicle.

Obtaining the second speed information of the second vehicle at the second time and at the second location comprises obtaining the second speed information using one or more sensors. Obtaining the second speed information using one or more sensors can include obtaining the second speed information using one or more of LIDAR, a monocular camera, a stereoscopic camera, or radar.

Methods can include determining a lane of travel for the second vehicle, wherein transmitting the second speed information of the second vehicle is conditioned on the lane of travel for the second vehicle being other than a right-most lane of travel.

Systems, methods, and computer program products are described for monitoring vehicles and detecting potential traffic violations through a user device at a user's vehicle. Example user devices include personal computers, mobile communication devices, and other devices that can acquire image data and can send and receive data over a network. The subject matter addresses the technical challenges of providing and implementing a vehicle monitoring application at a user device located within the vehicle that can be used within a vehicle monitoring system to determine if a second vehicle is violating a traffic violation.

The technologies described in this document can benefit any type of vehicle. In particular, technologies described in this document can benefit any type of vehicle that is used on road ways with one or more additional vehicles. For brevity, the description that follows refers to a particular road violation, e.g., detecting a slow vehicle in a left lane of a multilane roadway, but the description is also applicable to monitoring vehicle traffic and detecting other traffic violations.

These features and additional features are described in more detail below.

<FIG> is a block diagram of an example vehicle monitoring system <NUM>. The vehicle monitoring system <NUM> includes a network <NUM>, such as a local area network (LAN), a wide area network (WAN), the Internet, or a combination thereof. The network <NUM> connects a traffic information server <NUM>, a user device <NUM> through a network server <NUM>, an application server <NUM>, and a reporting agency server <NUM>. The vehicle monitoring system <NUM> may include many different traffic information servers <NUM>, user devices <NUM>, network servers <NUM>, application servers <NUM>, and reporting agency servers <NUM>. The application server <NUM> may represent a combination of application servers, database servers, communication servers, web servers, and the like that comprise the systems of the application provider used to collect data from and communicate with various user devices <NUM> through a vehicle monitoring application <NUM>.

The vehicle monitoring application <NUM> can be installed on and/or executed by a user device <NUM>. A user device <NUM> is an electronic device that is capable of sending and receiving data over a data communication network <NUM>. Example user devices <NUM> include smart phones, tablet computing devices, wearable computing devices (e.g., smart watches), and other devices that can send and receive data over the network <NUM>.

A user device <NUM> typically includes a user application to facilitate the sending and receiving of data over the network <NUM>. For example, web browsers or native applications executed by the user device <NUM> can facilitate the sending and receiving of data over the network <NUM>. An application executing on a user device <NUM> can include an application environment, e.g., a graphical user interface (GUI) <NUM>, in which images may be shown. Examples of such applications are camera-enabled applications that can use an on-device camera, such as camera <NUM>, to capture an image, applications that can receive images and display images, and applications that can access and display images that are stored on the user device <NUM>. For example, an application may have access to a repository of image data <NUM> stored on the mobile device <NUM>, where the application environment can load an image from the image data <NUM>. Each image of the image data <NUM> can include location data, a timestamp, or the like. In some implementations, the embedded data in the image data <NUM> can be used by the processes described herein.

The vehicle monitoring application <NUM> can be implemented as a native application developed for a particular platform or a particular device, a browser-based application that provides a web interface, or another appropriate type of application. The vehicle monitoring application <NUM> performs object recognition on an image or a plurality of image frames within an application environment on the mobile device <NUM>. For example, the vehicle monitoring application <NUM> can include an image recognition processor <NUM> that attempts to detect and recognize (e.g., identify) objects in pixel data (or images). The image recognition processor <NUM> can detect various objects, such as people, barcodes, landmarks, paintings, vehicles, and/or other types of objects using edge detection and/or other object recognition techniques. In particular, for the process of detecting a slow vehicle in a left lane of a roadway, the image recognition processor <NUM> can detect other vehicles and particular roadway features, such as the number of vehicular traffic lanes, lane width, lane traffic direction, and lane marker types that may signify a left lane (e.g., solid white line, double yellow lines, a guard rail, or a high occupancy vehicle (HOV) lane with double white lines). The image recognition processor <NUM> can access the displayed image data by means of an application programing interface (API).

The vehicle monitoring application <NUM> can perform tasks or services for a user of the mobile device <NUM>. For example, the vehicle monitoring application <NUM> can respond to voice commands of the user (e.g., start a vehicle monitoring process). To make it easier and more efficient for the user to implement a vehicle monitoring process, the vehicle monitoring application <NUM> can present controls on the GUI <NUM> of a display of the mobile device <NUM>. In some implementations, the vehicle monitoring application <NUM> can determine that the vehicle is in a driving mode, and automatically initiate a vehicle monitoring process by activating the camera of the user device, and continuously monitoring the image data to determine if the user is driving in the left lane and to detect if another vehicle is within the predetermined distance in front of the user's vehicle.

In operation, a camera <NUM> of the user device <NUM> is presented with a view in a forward direction relative to the user's vehicle <NUM>. In some implementations, the user device <NUM> can be coupled to the vehicle with a mounting device. In some implementations, a manufacturer of the user's vehicle <NUM> can include the vehicle monitoring application <NUM> in an on board computing system, and include one or more fixed cameras on the user's vehicle <NUM> such that the cameras can view the surrounding vicinity of the user's vehicle <NUM>.

The vehicle monitoring application <NUM> can attempt to detect roadway feature information and vehicles in image data received from a camera <NUM> of the mobile device <NUM>, e.g., continuously without receiving a user request to initiate a vehicle monitoring process. For example, the vehicle monitoring application <NUM> can detect and/or recognize objects in a viewfinder of a camera <NUM> of the mobile device <NUM> (based on the image data) and interpret the fact that the user device is pointing the camera <NUM> at a roadway as a request to initiate a vehicle monitoring process.

The vehicle monitoring application <NUM> includes a vehicle monitoring engine <NUM> that receives identified object data from the image recognition processor <NUM> to analyze identified objects in the image data and initiate a vehicle monitoring process. As used herein, the term engine refers to software and/or hardware that performs a set of tasks, such as a data processing apparatus that performs the operations discussed herein. For example, for the process of detecting a slow vehicle in a left lane of a roadway, the vehicle monitoring engine <NUM> receives identified object data from image recognition processor <NUM> and can determine that the user's vehicle is traveling in a left lane of a multi-lane roadway having at least two lanes for travel in a first direction. The vehicle monitoring engine <NUM> can then detect, based on the identified object data from image recognition processor <NUM> identifying a vehicle, whether a vehicle is traveling within a predetermined distance (e.g., approximately an average car length of a vehicle or some other predefined distance, such as <NUM> meters, <NUM> meters, or any appropriate distance) in front of the user's vehicle and in the left lane of the roadway. An example of detecting whether a vehicle is traveling within a predetermined distance is illustrated in <FIG>.

The vehicle monitoring engine <NUM> can initiate communication from the user device <NUM> to a plurality of servers, such as the application server <NUM>, the traffic information server <NUM>, or the reporting agency server <NUM>. For example, for the process of detecting a slow vehicle in a left lane of a roadway, if the vehicle monitoring engine <NUM> determines that another vehicle is traveling within a predetermined distance and within a left lane, the vehicle monitoring engine <NUM> can then initiate a process of obtaining speed information of the user's vehicle and the second vehicle by requesting the information from the traffic information server <NUM>. An example of a timing diagram of a method for how the vehicle monitoring engine <NUM> requests and receives speed information from the traffic information server <NUM> is illustrated in <FIG>.

The vehicle monitoring engine <NUM> can further initiate communication from the user device <NUM> to the plurality of servers, such as the application server <NUM>, the traffic information server <NUM>, or the reporting agency server <NUM>, in order to obtain particular roadway information, such as a speed limit. The vehicle monitoring engine <NUM> can use the obtained roadway information in the process of detecting a slow vehicle in a left lane of a roadway. For example, the vehicle monitoring engine <NUM> can compare the speed information of the second vehicle to the speed limit of the roadway at a location of the second vehicle. When the comparison reveals that the speed of the second vehicle at the first time is below a defined speed limit of the roadway at the location of the second vehicle, the vehicle monitoring engine <NUM> can trigger a potential violation procedure. Examples of a flow diagram of a process for executing a potential violation procedure is illustrated in <FIG> and <FIG>.

The application server <NUM> is shown being operatively connected to at least one database <NUM> for storing data, including application data. According to various implementations, the application server <NUM> may communicate with several devices and/or servers, including one or more user devices <NUM>, a network server <NUM>, a traffic information server <NUM>, and a reporting agency server <NUM>. According to some implementations, a traffic information server <NUM> may be a separate server, as shown in <FIG>, or maybe included as part of the application server(s) <NUM>. The traffic information server <NUM> is shown being operatively connected to a database <NUM> to store traffic information data. The reporting agency server <NUM> is shown being operatively connected to a database <NUM> to store reporting agency data.

The network <NUM> may include various networking technologies that connect the network server <NUM> to the application server <NUM>, the traffic information server <NUM>, the reporting agency server <NUM>, including (among others) cellular data networks, Wi-Fi or WiMAX networks, satellite communication networks, metropolitan-area networks (MANs), wide-area networks (WANs), the Internet (TCP/IP), etc. The backhaul system of the network, the connection between the user device <NUM> and the network server <NUM>, may comprise ethernet, cellular (<NUM>, <NUM>, <NUM> LTE, etc.), Wi-Fi or other wireless local area network (LAN) (IEEE <NUM>), wired LAN (IEEE <NUM>), satellite phone (IRIDIUM), wireless personal area network (WPAN) (IEEE <NUM>), or any other telecommunications link, wired or wireless, including those identified above with regard to network <NUM>. As shown in <FIG>, one implementation includes user device <NUM> connected through a cellular network <NUM> via communication links <NUM>.

Processes and examples for detecting a slow vehicle in a left lane of a roadway are now described with reference to <FIG> below.

<FIG> is an illustration of an example roadway scenario. In particular, <FIG> displays a slower driver in the left lane, perpetrator vehicle <NUM>, causing traffic delays or traffic congestion, as a plurality of vehicles <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> are driving along in a same direction on a multi-lane roadway <NUM>. For example, the user's vehicle <NUM> is attempting to pass vehicle's <NUM> and <NUM>, which are driving in the middle lane, by driving at a faster speed in the left lane. However, perpetrator vehicle <NUM> is driving slower than the user's vehicle <NUM> causing the user to slow down. When a vehicle is caused to slow down in the left lane, this can cause a ripple effect of traffic congestion. For example, when slow drivers linger in the left lane of multi-lane roads, a driver may be forced to pass on the right, causing confusion and disorganization that can lead to accidents. Similarly, when a driver passes on the right, it may require other users to slow down. For example, assume that the vehicle <NUM> was previously driving in the left lane of <FIG> until vehicle <NUM> approached vehicle <NUM>. Further assume that as the vehicle <NUM> approached the vehicle <NUM>, vehicle <NUM> changed lanes from the left lane to the middle lane. In this example, when the vehicle <NUM> changed lanes into the center lane, this could have caused vehicle <NUM> to slow down, which would then have the ripple effect of slowing traffic in the center land and possibly the right lane as well. These type of incidents, which can be caused by violation of a slow poke law, can lead to aggressive driving by multiple drivers, which can lead to traffic congestions and vehicular accidents.

<FIG> is an illustration depicting a view captured by a user device. In this example, the user device <NUM> is shown as capturing image data (e.g., via camera <NUM>) and detecting whether the user's vehicle <NUM> is traveling in the left lane <NUM> of roadway <NUM>. For this particular roadway <NUM>, the far left lane <NUM> is marked by a solid line <NUM> on the left side, and a dashed line <NUM> on the right side, relative to the user's vehicle <NUM>. As the vehicle monitoring system acquires the image data, a determination can continuously (or periodically) be made as to whether the user's vehicle is traveling in a left lane <NUM> of the roadway <NUM>. According to some implementations, other types of sensors may be used to detect that the user's vehicle <NUM> is traveling in the left lane.

As illustrated, the shaded area <NUM> denotes the portion of the roadway <NUM> that is observed by the vehicle monitoring system. This portion of the roadway can be captured in the form of image data. The image data can include image data representing a view portion <NUM> that is captured on the right side of the solid line <NUM>. The image data can also include image data representing a different view portion <NUM> that is captured on the left hand side of the solid line <NUM>. In some situations, the image data for the view portion <NUM> can be used to identify one or more roadway features, such as signs or other features that are located to the left of the solid line <NUM>. In some situations, the image data for the view portion <NUM> can be ignored and/or discarded. For example, when the vehicle monitoring system is determining whether a second vehicle is ahead of the user's vehicle, as further discussed herein for <FIG>, the image data for the view portion <NUM> can be ignored.

As the vehicle monitoring system acquires the image data from the field of view of the camera on the mobile device <NUM>, a repository of image data <NUM> is stored on the mobile device <NUM>. For example, while the user's vehicle <NUM> is in operation, a collection of image data can be stored. The image data can be used in real time for the operation of the vehicle monitoring system described herein. According to some implementations, the image data can be stored and accessed at a later time by the vehicle monitoring application <NUM>, or application server <NUM>. For example, if the vehicle monitoring system is being used to collect data on particular vehicle that is driving erratically in front of the user's vehicle, then all of the image data with the erratic vehicle in the field of view of the camera can be accessed and sent to a reporting agency server <NUM>.

<FIG> and <FIG> illustrate various aspects of an example of a detection process. In particular, <FIG> shows an example with a second vehicle <NUM> in front of a user's vehicle <NUM>, but at a distance <NUM> greater than a predetermined distance <NUM>. <FIG> shows an example with a second vehicle in front of a user's vehicle <NUM>, but at a distance <NUM> that is equal to or less than the predetermined distance <NUM>. The shaded area <NUM> denotes the portion of the roadway <NUM> that is observed by the vehicle monitoring system using the image data. The predetermined distance <NUM> is used by the vehicle monitoring system to determine whether the second vehicle <NUM> (or another vehicle) is too close to the user's vehicle <NUM>. For example, the predetermined distance <NUM> can be set at <NUM> meters, which is approximately twice an average car length or some other appropriate distance. In the present example, <NUM> meters is used as an example of a distance at which a user is likely to begin to slow down due to the second vehicle <NUM> driving at slower speeds. Alternatively, other distances can be used for the predetermined distance <NUM>.

<FIG> is a flowchart of an example process <NUM> for triggering a potential violation procedure in a vehicle monitoring system. The process <NUM> can be implemented, for example, by the vehicle monitoring system <NUM> of <FIG>. Operations of the process <NUM> can also be implemented as instructions stored on non-transitory computer readable media, and execution of the instructions be one or more data processing apparatus can cause the one or more data processing apparatus to perform the operations of the process <NUM>.

The process <NUM> acquires image data that depicts a view in a forward direction relative to the user's vehicle (<NUM>). The image data can be acquired by a camera of a user's device. For example, as shown in <FIG>, camera <NUM> of user device <NUM> can be used to acquire the image data, where user device <NUM> is located within or attached to user's vehicle <NUM>. As shown in <FIG>, a camera view of the user device <NUM> is shown facing forward relative to the vehicle <NUM>.

The process <NUM> determines that the user's vehicle is traveling in a left lane of a roadway having at least two lanes for travel in a first direction (<NUM>). For example, as shown in <FIG>, assume that the user's vehicle <NUM> is traveling in the left lane <NUM> of a multi-lane roadway <NUM>. In this example, the vehicle monitoring application <NUM> on the user device <NUM> is able to determine the vehicle <NUM> is in the left lane based on the image data. In some implementations, object recognition can be performed on the image data to determine that the user's vehicle is traveling in the left lane of the roadway. For example, the object recognition can enable the vehicle monitoring application <NUM> to determine that a particular roadway feature indicative of travel in the left lane is positively identified in the image data. In some situations, the particular roadway feature that leads to the determination that the user's vehicle is traveling in the left lane can include, but is not limited to, a solid white line, double yellow lines, a guard rail, a high occupancy vehicle (HOV) lane with double white lines, or the like. For instance, when the object recognition identifies one of these particular roadway features at specific locations in the image data, the conclusion can be made that the user's vehicle is traveling in the left lane. More specifically, the vehicle monitoring application <NUM> can conclude that the user's vehicle is traveling in the left lane when any of the roadway features listed above are determined to be located on a left side of the image captured by the user device <NUM>.

The process <NUM> detects that a second vehicle is traveling within a predetermined distance in front of the user's vehicle and in the left lane of the roadway (<NUM>). The image recognition processor <NUM> can analyze the image data to detect that one or more vehicles are present within the viewing area of the camera. The vehicle monitoring engine <NUM> can then determine, based on the image data, that a second vehicle is in the left lane in front of the user's vehicle <NUM>, and determine the distance between the two vehicles. For example, as shown in <FIG>, the closest portion of the perpetrator's vehicle <NUM> (e.g., the rear bumper) relative to the front portion of the user's vehicle <NUM> (e.g., the front bumper), is equal to or less than the predetermined distance <NUM>. Thus, the image recognition processor <NUM> detected a vehicle, perpetrator's vehicle <NUM>, within the viewing area of the camera, shown as shaded area <NUM>. The vehicle monitoring engine <NUM> can analyze the image data with the detected vehicle to determine that the perpetrator's vehicle <NUM> is within the predetermined distance <NUM>.

The process <NUM> obtains speed information of the user's vehicle and the second vehicle at a first time and at a first location (<NUM>). The vehicle monitoring engine <NUM> can initiate a process of obtaining speed information of the user's vehicle and the second vehicle. For example, the system can obtain and use a current speed of the user's vehicle <NUM> through communication with the user's vehicle <NUM> on board computing system. After obtaining the current speed, the system is adapted to use the image data and determine that a change in distance over time relative to the other car to determine the speed of that other car. For example, <FIG> shows the perpetrator's vehicle <NUM> at distance <NUM> at a first time, and <FIG> shows the perpetrator's vehicle <NUM> at a second distance <NUM> at a second time. The vehicle monitoring system can then determine the approximate speed of the perpetrator's vehicle <NUM>. In accordance with the invention, the vehicle monitoring system is adapted to determine a difference in the distance <NUM> between the vehicle <NUM> and the vehicle <NUM> between the first time and the second time. That difference in distance in combination with the amount of time that elapsed between the first time and the second time can be used to compute the difference in distance over time. That difference in distance over time can then be converted into an equivalent difference in miles per hour or kilometers per hour, which can represent the difference in speed between the two vehicles <NUM> and <NUM>. This difference in speed can be used along with the speed of the vehicle <NUM> to determine the speed of the vehicle <NUM>. More specifically, assume that over a <NUM> second interval, vehicle <NUM> gains <NUM> on the vehicle <NUM> (e.g., meaning that the distance difference between the first time and the second time is <NUM>). Thus, the vehicle <NUM> is traveling <NUM>/hour faster than the vehicle <NUM>. Assuming that the vehicle <NUM> is traveling at <NUM>/hour, this means that the vehicle <NUM> is traveling <NUM> per hour. If the speed limit is <NUM>/hour, the vehicle <NUM> is travelling well under the speed limit.

In some implementations, the system may obtain speed information by requesting the information from the traffic information server <NUM>. An example of a timing diagram of a process for requesting and receiving speed information from the traffic information server <NUM> is illustrated in <FIG>.

According to the invention, the system obtains speed information from one or more sensors (e.g., LIDAR, monocular or stereoscopic cameras, and/or RADAR) connected to or within the vehicle in lieu of, or in addition to, obtaining speed information from the image data or a traffic information server. The one or more sensors may be mounted to or attached to the vehicle. Additionally, or alternatively, the speed information may be extrapolated by the user device by using communications with the user's vehicle. For example, if the user's vehicle maintains the same distance behind the second vehicle and is traveling <NUM> mph, then the vehicle monitoring system can then determine the second vehicle is traveling at that same speed.

The process <NUM> determines that a speed of the second vehicle at the first time is below a defined speed limit of the roadway at the first location based on the obtained speed information (<NUM>). For example, the speed information obtained from the traffic information server <NUM>, can also include the defined speed limit of the roadway at the particular location the speed information was obtained. Other processes of obtaining a speed limit may be used by the vehicle monitoring application <NUM>. For example, the vehicle monitoring application <NUM> may be able to access GPS mapping application data stored on the user's device. Additionally, the vehicle monitoring application <NUM> may request the information from the application server <NUM> or traffic information server <NUM>. In some implementations, the image recognition processor <NUM> could analyze the image data for roadway signs that display the speed limit at specific locations the speed information is obtained.

The process <NUM> triggers a potential violation procedure (<NUM>). According to the example implementations, the potential violation is a violation of a slow poke law, i.e., driving too slow in a left lane of a multi-lane roadway. Alternatively or additionally, other violation procedures may be used using the vehicle monitoring system <NUM>. <FIG> and <FIG> each provide an example process for executing a potential violation procedure of step <NUM> detecting a slow vehicle in a left lane of a roadway.

<FIG> is a flowchart of an example process for executing a potential violation procedure in a vehicle monitoring system. The process <NUM> can be implemented, for example, by the vehicle monitoring system <NUM> of <FIG>. Operations of the process <NUM> can also be implemented as instructions stored on non-transitory computer readable media, and execution of the instructions be one or more data processing apparatus can cause the one or more data processing apparatus to perform the operations of the process <NUM>.

The process <NUM> detects that a second vehicle is traveling within a predetermined distance in front of a user's vehicle and in the left lane of a roadway after a predetermined amount of time (<NUM>). For example, as shown in <FIG>, the perpetrator's vehicle <NUM> is within the predetermined distance <NUM>. According to the process <NUM>, the vehicle monitoring application <NUM> will wait for a predetermined amount of waiting time, e.g. thirty seconds, and again determine if the second vehicle is still within the predetermined distance <NUM>. When the perpetrator's vehicle <NUM> is still within the predetermined distance <NUM> after the predetermined amount of waiting time, the process <NUM> continues to <NUM>. The predetermined amount of time may be any time interval, e.g., <NUM> seconds, <NUM> seconds, <NUM> minute, or any other time interval set within or by the vehicle monitoring application. This delay allows a perpetrator vehicle, e.g., a driver temporarily driving slowly in the left lane, the opportunity to move over and out of the left lane such that a false positive is not triggered by the vehicle monitoring application <NUM>. When the vehicle monitoring application <NUM> determines that the second vehicle is no longer within the predetermined distance <NUM> after the predetermined amount of waiting time, the potential violation procedure can terminate without reporting the potential violation.

Speed information of the user's vehicle and the second vehicle are obtained at a second time at a second location (<NUM>).

The speed of the second vehicle at the second time is determined (<NUM>). Obtaining the speed information and the defined speed limit of the roadway at the second location may be similar to the steps discussed herein for steps <NUM> and <NUM>, respectively. However, the additional speed information is now obtained for an additional point in time, i.e., after the predetermined amount of time, and at a second location. The vehicle monitoring application <NUM> can determine that the speed of the second vehicle is below a defined speed limit of the roadway at the second location based on a comparison of the speed information and the defined speed limit of the roadway. When the vehicle monitoring application <NUM> determines that the speed of the second vehicle is below the defined speed limit of the roadway, the process <NUM> can proceed to <NUM>. When the vehicle monitoring application <NUM> determines that the speed of the second vehicle is above the defined speed limit of the roadway, the process <NUM> can terminate without reporting the potential violation.

A plate number of the second vehicle is detected based on image data (<NUM>). According to some implementations, the plate number can be detected from the image data by an image recognition process, such as optical character recognition (OCR) or the like, performed by the image recognition processor <NUM> at the user device <NUM>. In some implementations, the image data can be sent to and processed by the application server <NUM> to detect the plate number. According to some implementations, detecting the plate number can include detecting the plate number from a first image or a subsequently acquired image when the plate number is not detected in the first image. For example, the vehicle monitoring system <NUM> can determine whether the first image acquired of the plate of the second vehicle displays a readable image, or if an additional image needs to be acquired in order to recognize the plate number.

The process <NUM> transmits the speed information of the second vehicle at the first time, the speed information of the second vehicle at the second time, the plate number of the second vehicle, and the image data to an application server (<NUM>). For example, as shown in <FIG>, the user device <NUM>, using the vehicle monitoring application <NUM>, can send user device data (e.g., the speed information of the second vehicle at the first time, the speed information of the second vehicle at the second time, the plate number of the second vehicle, and the image data, and the like) through the network server <NUM> to the application server <NUM> over the network <NUM>.

The process <NUM> detects that a second vehicle is traveling within a predetermined distance in front of a user's vehicle and in the left lane of a roadway after a predetermined amount of time (<NUM>). For example, as shown in <FIG>, the perpetrator's vehicle <NUM> is within the predetermined distance <NUM>. According to the process <NUM>, the vehicle monitoring application <NUM> will wait for a predetermined amount of waiting time, e.g. thirty seconds, and again determine if the second vehicle is still within the predetermined distance <NUM>. When the perpetrator's vehicle <NUM> is still within the predetermined distance <NUM> after the predetermined amount of waiting time, the process <NUM> continues to <NUM>. The predetermined amount of time may be any time interval, e.g., <NUM> seconds, <NUM> seconds, <NUM> minute, or any other time interval set within or by the vehicle monitoring application. As discussed herein, this delay can be any amount of time the vehicle monitoring system <NUM> has implemented for the specific application being used. For detecting a slower driver in a left lane, at least some amount of delay may be necessary to reduce the amount of violations that may only be occurring for short amount of time, e.g., less than five seconds, which may not be impacting traffic. When the vehicle monitoring application <NUM> determines that the second vehicle is no longer within the predetermined distance <NUM> after the predetermined amount of waiting time, the potential violation procedure can terminate without reporting the potential violation.

Speed information of a first set of vehicles traveling in the first direction of the roadway that are within a defined area around the first location are obtained after a predetermined amount of time (<NUM>). For example, <FIG> shows an example detection process of an example road scenario with a shaded area <NUM> with an arcuate shape around the center of user's vehicle <NUM>. Obtaining the speed information of a first set of vehicles may include each vehicle <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> traveling in the same direction of the user's vehicle and are each within the shaded area <NUM>. In some implementations, obtaining the speed information is sent by request to the traffic information server <NUM>. Using GPS coordinates, or the like, the traffic information server <NUM> sends the speed information for each vehicle within the defined area.

According to some implementations, a sensor, such as a LIDAR, may be used to obtain the speed information. For example, <FIG> shows an example detection process of an example road scenario with a user's vehicle equipped with a sensor <NUM> used to detect speed information of surrounding objects, such as vehicles <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> traveling in the same direction of the user's vehicle. The shaded area <NUM> represents the coverage area that the sensor <NUM> is able to detect and obtain speed information of all surrounding objects. As shown in <FIG>, shaded area <NUM> has an arcuate shape around the sensor <NUM>, but shaded area <NUM> is not extending past the left lane because the vehicle monitoring system is obtaining speed information for vehicles traveling in the same direction of the user's vehicle <NUM>. Thus, only vehicles in the left lane and in the lanes to the right of the user's vehicle <NUM>.

A LIDAR device, such as sensor <NUM>, can actively estimate distances to objects in an environment, such as vehicles <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> in <FIG>. A LIDAR can scan a particular area of an environment, such as shaded area <NUM>, and accumulate a plurality of point positions indicative of a three-dimensional shape, such as a vehicle. For example, individual points can be measured by generating a laser pulse and detecting a returning pulse, if any, reflected from an object, and determining the distance to the reflective object according to the time delay between the emitted pulse and the reception of the reflected pulse. The laser, or set of lasers, can be rapidly and repeatedly used across an environment to provide continuous distances to reflective objects in the scene. A three-dimensional map of points of reflective features can be generated based on returning pulses. The three-dimensional point map can indicate positions of reflective objects in the scanned environment. To detect a speed of a vehicle in the shaded area <NUM>, a plurality of target points that correspond to a target surface of a target vehicle can be identified in the set of spatial points. The plurality of target points can include a first point indicative of a first location on the target surface obtained by the LIDAR device at a first time and a second point indicative of a second location on the target surface obtained by the LIDAR device at a second time. A speed of the target vehicle can be estimated based on the first location, the first time, the second location, and the second time. For example, the difference between the target vehicle's location over the time interval can be used along with the speed of the user's vehicle can be used to determine the speed of the target vehicle in a manner similar to that described above.

Speed information of a second set vehicles traveling in the first direction of the roadway that are within a defined area around a second location are obtained after the predetermined amount of time (<NUM>). According to the process <NUM>, the vehicle monitoring application <NUM> will wait for a predetermined amount of waiting time, and then obtain an additional data set of speed information. For example, after thirty seconds, if the perpetrator's vehicle <NUM> is still within the predetermined distance <NUM>, then process <NUM> continues. The predetermined amount of time may be any time interval, e.g., <NUM> seconds, <NUM> seconds, <NUM> minute, or any other time interval set by the vehicle monitoring application. This time delay allows a perpetrator vehicle, e.g., a driver temporarily driving slowly in the left lane, the opportunity to move over and out of the left lane such that a false positive is not triggered by the vehicle monitoring application <NUM>. When the vehicle monitoring application <NUM> determines that the second vehicle is no longer within the predetermined distance <NUM> after the predetermined amount of waiting time, the potential violation procedure can terminate without reporting the potential violation.

A determination is made that a traffic delay is occurring and is caused by the second vehicle based on the speed information of the first set of vehicles and the second set of vehicles (<NUM>). For example, the vehicle monitoring application <NUM> can send all of the information obtained from the first and second set of vehicles and send that information to the application server <NUM> to analyze and determine whether traffic delays were caused by the perpetrator vehicle <NUM>. When the vehicle monitoring application <NUM> determines that there is a traffic delay and is caused by the second vehicle, the process <NUM> can proceed to <NUM>. When the vehicle monitoring application <NUM> determines that there is no traffic delay, or the traffic delay is not caused by the second vehicle, the process <NUM> can terminate without reporting the potential violation.

In response to determining the traffic delay has occurred, traffic delay information is generated (<NUM>). The traffic delay information can include the speed information of the first set of vehicles and the second set of vehicles, the speed information of the second vehicle at the first time, the speed information of the second vehicle at the second time, the plate number of the second vehicle, and the image data. For example, the application server <NUM> determines whether traffic delays were caused by the perpetrator vehicle <NUM> based on the analysis of the obtained speed information from the user device <NUM>. In some implementations, the vehicle monitoring application <NUM> operating on the user device can make the determination that traffic delays were caused by the perpetrator vehicle <NUM>.

Traffic delay information is transmitted to a reporting agency server (<NUM>). For example, if the application server <NUM> determines that traffic delays were caused by the perpetrator vehicle <NUM>, the application server <NUM> can send the traffic delay information to a reporting agency server <NUM>. For example, <FIG> illustrates a timing diagram of a method for exchanging data in an example vehicle monitoring system, including sending user device data to the application server <NUM>, and sending traffic delay information to a reporting agency server <NUM>.

<FIG> illustrates a timing diagram <NUM> of a method for exchanging data in an example vehicle monitoring system, such as vehicle monitoring system <NUM> of <FIG>, according to examples of the present disclosure. As described herein, for some implementations, the vehicle monitoring application <NUM> may obtain speed information from a traffic information server, such as traffic information server <NUM>. The timing diagram <NUM> illustrates these type of communication messages for obtaining speed information. Additionally, or alternatively, timing diagram <NUM> illustrates sending communication messages, such as traffic delay information, from a user device <NUM> to a reporting agency server, such as reporting agency server <NUM>.

As shown in <FIG>, the user device <NUM> sends a message packet with a speed information request to the network server <NUM> through the network server provider's backhaul, and then the network server <NUM> sends the request message to the application server <NUM> over the network, such as network <NUM> (e.g., via TCP/IP). The application server <NUM> then sends the request message to the traffic information server <NUM> over the network. The traffic information server <NUM> then receives the speed information request and sends a response with the applicable speed information to the application server <NUM>. The application server <NUM> then sends the speed information to the user device <NUM> through the network server <NUM>. This process of obtaining speed information from the traffic information server <NUM> may be repeated, as necessary, to process a potential violation procedure as described herein.

Additionally, as described in <FIG>, the vehicle monitoring system can compile traffic delay information and send the information to a reporting agency, who can in turn, report the traffic delays caused by the driver or owner of the perpetrator's vehicle and/or issue a citation for a violation, if applicable. For example, as shown in <FIG>, the user device <NUM> sends a message packet with user device data (e.g., the obtained traffic information) to the network server <NUM> through the network server provider's backhaul, and then the network server <NUM> sends the data message to the application server <NUM> over the network. The application server <NUM> analyzes and can also validate the user device data to determine that a traffic delay was caused by the perpetrator's vehicle and/or a violation occurred, and the application server <NUM> can send the validated user device data and compiled traffic delay information to the reporting agency server <NUM> over the network.

In some implementations, speed information may be obtained for an additional set of vehicles traveling in the first direction of the roadway that are within a defined area around the first location after an additional predetermined amount of time. The vehicle monitoring system <NUM> may use this additional information for additional traffic delay information to provide to a driver of a perpetrator vehicle <NUM> that would illustrate that the driver's slow driving in the left lane at the first location, caused vehicles to travel a lower speed at that same first location several minutes later.

In some implementations, one or more sensing devices are connected to the vehicle for monitoring a plurality of other vehicles surrounding the user's vehicle and a perpetrator's vehicle when the perpetrator's vehicle and the other vehicles are within the range of the sensing devices. The one or more sensing devices can monitor surrounding vehicles located on every side of the user's vehicle, and the vehicle monitoring application can identify cars passing on each side of the vehicle. Additionally, or alternatively, the vehicle monitoring application <NUM> on the user device can determine other violations or other issues caused by slower drivers. These other violations may not solely correspond to violations occurring in a left lane. For example, the vehicle monitoring system can detect that vehicles are passing the perpetrator's vehicle (e.g., the vehicle moving too slowly) on the right side of the perpetrator's vehicle of a multi-lane highway. Further, the user's vehicle is not necessarily in the far left lane to detect the slower vehicle, because even in a middle lane, the slower driver may be causing larger vehicles, such as semi-trucks, to enter the left lane to pass the slower vehicle. Semi-trucks moving into the left lane can cause issues. The vehicle monitoring system <NUM> may use this additional information for additional traffic delay information to provide to a driver of a perpetrator vehicle <NUM> that would illustrate that the driver's slow driving in the left lane at the first location, caused vehicles to travel a lower speed at that same first location several minutes later.

<FIG> is a block diagram of example computing devices <NUM>, <NUM> that can be used to implement the systems and methods described in this document, as either a client or as a server or plurality of servers. Computing device <NUM> is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Computing device <NUM> is further intended to represent any other typically non-mobile devices, such as televisions or other electronic devices with one or more processers embedded therein or attached thereto. Computing device <NUM> is intended to represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, and other computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosures described and/or claimed in this document.

Computing device <NUM> includes a processor <NUM>, memory <NUM>, a storage device <NUM>, a high-speed controller <NUM> connecting to memory <NUM> and high-speed expansion ports <NUM>, and a low-speed controller <NUM> connecting to low-speed bus <NUM> and storage device <NUM>. The processor <NUM> can process instructions for execution within the computing device <NUM>, including instructions stored in the memory <NUM> or on the storage device <NUM> to display graphical information for a graphical user interface (GUI) on an external input/output device, such as display <NUM> coupled to high-speed controller <NUM>.

The high-speed controller <NUM> manages bandwidth-intensive operations for the computing device <NUM>, while the low-speed controller <NUM> manages lower bandwidth-intensive operations. Such allocation of duties is an example only. In the implementation, low-speed controller <NUM> is coupled to storage device <NUM> and low-speed bus <NUM>. The low-speed bus <NUM> (e.g., a low speed expansion port), which may include various communication ports (e.g., USB, Bluetooth®, Ethernet, wireless Ethernet), may be coupled to one or more input/output devices, such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or router, e.g., through a network adapter.

Alternatively, components from computing device <NUM> may be combined with other components in a mobile device (not shown), such as computing device <NUM>. Each of such devices may contain one or more of computing devices <NUM>, <NUM>, and an entire system may be made up of multiple computing devices <NUM>, <NUM> communicating with each other.

The computing device <NUM> may also be provided with a storage device, such as a micro-drive or other device, to provide additional storage.

The processor <NUM> can process instructions for execution within the computing device <NUM>, including instructions stored in the memory <NUM>. The processor may also include separate analog and digital processors. The processor may provide, for example, for coordination of the other components of the computing device <NUM>, such as control of user interfaces, applications run by computing device <NUM>, and wireless communication by computing device <NUM>.

The display <NUM> may be, for example, a TFT LCD display or an OLED display, or other appropriate display technology. In addition, an external interface <NUM> may be provided in communication with processor <NUM>, so as to enable near area communication of computing device <NUM> with other devices. External interface <NUM> may provide, for example, for wired communication (e.g., via a docking procedure) or for wireless communication (e.g., via Bluetooth® or other such technologies).

Expansion memory <NUM> may also be provided and connected to computing device <NUM> through expansion interface <NUM>, which may include, for example, a subscriber identification module (SIM) card interface. Such expansion memory <NUM> may provide extra storage space for computing device <NUM>, or may also store applications or other information for computing device <NUM>. Thus, for example, expansion memory <NUM> may be provide as a security module for computing device <NUM>, and may be programmed with instructions that permit secure use of computing device <NUM>. In addition, secure applications may be provided via the SIM cards, along with additional information, such as placing identifying information on the SIM card in a non-hackable manner.

The memory may include for example, flash memory and/or MRAM memory, as discussed below. The information carrier is a computer- or machine-readable medium, such as the memory <NUM>, expansion memory <NUM>, or memory on processor <NUM>.

Computing device <NUM> may communicate wirelessly through communication interface <NUM>, which may include digital signal processing circuitry where necessary. Such communication may occur, for example, through transceiver <NUM> (e.g., a radio-frequency transceiver). In addition, short-range communication may occur, such as using a Bluetooth®, WiFi, or other such transceiver (not shown). In addition, GPS receiver module <NUM> may provide additional wireless data to computing device <NUM>, which may be used as appropriate by applications running on computing device <NUM>.

Computing device <NUM> may also communicate audibly using audio codec <NUM>, which may receive spoken information from a user and convert it to usable digital information. Audio codec <NUM> may likewise generate audible sound for a user, such as through a speaker, e.g., in a handset of computing device <NUM>. Such sound may include sound from voice telephone calls, may include recorded sound (e.g., voice messages, music files, etc.) and may also include sound generated by applications operating on computing device <NUM>.

It may also be implemented as part of a smartphone <NUM>, personal digital assistant, or other mobile device.

In situations in which the systems discussed here collect personal information about users, or may make use of personal information, the users may be provided with an opportunity to control whether applications or features collect user information (e.g., information about a user's social network, social actions or activities, profession, a user's preferences, or a user's current location), or to control whether and/or how to receive content that may be more relevant to the user. In addition, certain data may be treated in one or more ways before it is stored or used, so that personally identifiable information is removed. For example, a user's identity may be treated so that no personally identifiable information can be determined for the user, or a user's geographic location may be generalized where location information is obtained (such as to a city, ZIP code, or state level), so that a particular location of a user cannot be determined. Thus, the user may have control over how information is collected about the user and used by a content server.

Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on computer storage medium for execution by, or to control the operation of, data processing apparatus.

Moreover, while a computer storage medium is not a propagated signal, a computer storage medium can be a source or destination of computer program instructions encoded in an artificially generated propagated signal.

The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., a FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks.

In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user's user device in response to requests received from the web browser.

Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back end, middleware, or front end components.

The computing system can include users and servers. A user and server are generally remote from each other and typically interact through a communication network. The relationship of user and server arises by virtue of computer programs running on the respective computers and having a user-server relationship to each other. In some embodiments, a server transmits data (e.g., an HTML page) to a user device (e.g., for purposes of displaying data to and receiving user input from a user interacting with the user device). Data generated at the user device (e.g., a result of the user interaction) can be received from the user device at the server.

While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any features or of what may be claimed, but rather as descriptions of features specific to particular embodiments.

Thus, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In certain implementations, multitasking and parallel processing may be advantageous.

Claim 1:
A method for detecting a slow vehicle in a roadway, the method comprising:
acquiring (<NUM>), from a camera of a user device (<NUM>), image data that depicts a view in a forward direction relative to a first vehicle, the user's vehicle (<NUM>), wherein the acquired image data represents an image of a second vehicle (<NUM>) that is traveling in front of the first vehicle;
obtaining (<NUM>) first speed information of the second vehicle at a first time and at a first location, wherein obtaining the first speed information of the second vehicle comprises obtaining a current speed of the first vehicle, determining based on said image data of the user device, a change in distance over time relative to the second vehicle, and determining the first speed information of the second vehicle based on the current speed of the first vehicle and on the change in distance over time;
determining (<NUM>), based on the obtained speed information, that a speed of the second vehicle at the first time is below a defined speed limit of the roadway at the first location;
detecting, after a predetermined amount of time, that the second vehicle is traveling within a predetermined distance (<NUM>) in front of the first vehicle;
obtaining (<NUM>) second speed information of the second vehicle at a second time and at a second location, wherein obtaining the second speed information of the second vehicle at the second time and at the second location comprises obtaining the second speed information using one or more sensors connected to or within the first vehicle, and wherein the one or more sensors exclude the camera of the user device;
determining (<NUM>), based on the obtained second speed information, that the speed of the second vehicle at the second time is below the defined speed limit of the roadway at the second location; and
transmitting, to an application server (<NUM>), the second speed information of the second vehicle at the second time and the image data.