Enhanced vehicle operation

A computer includes a processor and a memory, the memory storing instructions executable by the processor to collect steering, speed, and position data about a plurality of vehicles from one or more infrastructure sensors, identify a vehicle that varies from a specified position in a roadway lane relative to a roadway lane marker or exceeds a threshold speed based on the collected data, instruct the identified vehicle to move to a side of a roadway, and send a message to a central server including an identification of the vehicle.

BACKGROUND

Vehicles can collect data of their surroundings while operating. Based on the data, computers in the vehicles can identify nearby objects. For example, the computers can detect other vehicles traveling on the roadway. The computers can transmit the data to a central server. The transmissions occur over a network. Such networks typically have dedicated bandwidth for transmissions.

DETAILED DESCRIPTION

A computer includes a processor and a memory, the memory storing instructions executable by the processor to collect steering, speed, and position data about a plurality of vehicles from one or more infrastructure sensors, identify a vehicle that varies from a specified position in a roadway lane relative to a roadway lane marker or exceeds a threshold speed based on the collected data, instruct the identified vehicle to move to a side of a roadway, and send a message to a central server including an identification of the vehicle.

The instructions can further include instructions to identify the vehicle when the speed of one of the plurality of vehicles exceeds a posted speed limit.

The instructions can further include instructions to identify the vehicle when one of the plurality of vehicles moves toward a first roadway lane marking and then moves toward a second roadway lane marking laterally disposed from the first roadway lane marking.

The instructions can further include instructions to identify the vehicle when one of the plurality of vehicles moves into a portion of a roadway where vehicles are not permitted to operate.

The instructions can further include instructions to collect data about the plurality of vehicles from one or more of the plurality of vehicles.

The instructions can further include instructions to instruct the vehicle to power off.

The instructions can further include instructions to input the collected data to a machine learning program to provide an output identifying the vehicle.

The machine learning program can be trained with previously collected vehicle data.

The computer can be disposed at an intersection and the instructions can further include instructions to collect data about the plurality of vehicles from a second computer disposed at a second intersection.

The intersection can include at least one roadway sign or roadway light, at least one of the infrastructure sensors is disposed on the roadway sign or the roadway light, and the instructions can further include instructions to collect data from the at least one infrastructure sensor disposed on the roadway sign or the roadway light at the intersection.

The instructions can further include instructions to receive an identification of the vehicle from one of the plurality of vehicles.

The infrastructure sensors can be disposed on at least one of a roadway sign or a roadway light.

A method includes collecting steering, speed, and position data about a plurality of vehicles from one or more infrastructure sensors, identifying a vehicle that varies from a specified position in a roadway lane relative to a roadway lane marker or exceeds a threshold speed based on the collected data, instructing the identified vehicle to move to a side of a roadway, and sending a message to a central server including an identification of the vehicle.

The method can further include identifying the vehicle when the speed of one of the plurality of vehicles exceeds a posted speed limit.

The method can further include identifying the vehicle when one of the plurality of vehicles moves toward a first roadway lane marking and then moves toward a second roadway lane marking laterally disposed from the first roadway lane marking.

The method can further include identifying the vehicle when one of the plurality of vehicles moves into a portion of a roadway where vehicles are not permitted to operate.

The method can further include collecting data about the plurality of vehicles from one or more of the plurality of vehicles.

The method can further include instructing the vehicle to power off.

The method can further include inputting the collected data to a machine learning program to provide an output identifying the vehicle.

The method can further include collecting data about the plurality of vehicles from a computer disposed at an intersection.

The intersection can include at least one roadway sign or roadway light, at least one of the infrastructure sensors is disposed on the roadway sign or the roadway light, and the method can further include collecting data from the at least one infrastructure sensor disposed on the roadway sign or the roadway light at the intersection.

The method can further include receiving an identification of the vehicle from one of the plurality of vehicles.

A system includes a central server, one or more infrastructure sensors, a plurality of vehicles, a local server in communication with the central server, the infrastructure sensors, and the plurality of vehicles, means for collecting steering, speed, and position data about the plurality of vehicles from the infrastructure sensors, means for identifying a vehicle that varies from a specified position in a roadway lane relative to a roadway lane marker or exceeds a threshold speed based on the collected data, means for instructing the identified vehicle to move to a side of a roadway, and means for sending a message to the central server including an identification of the vehicle.

The system can further include means for identifying the vehicle when the speed of one of the plurality of vehicles exceeds a posted speed limit.

The system can further include means for identifying the vehicle when one of the plurality of vehicles moves toward a first roadway lane marking and then moves toward a second roadway lane marking laterally disposed from the first roadway lane marking.

The system can further include means for identifying the vehicle when one of the plurality of vehicles moves into a portion of a roadway where vehicles are not permitted to operate.

Further disclosed is a computing device programmed to execute any of the above method steps. Yet further disclosed is a vehicle comprising the computing device. Yet further disclosed is a computer program product, comprising a computer readable medium storing instructions executable by a computer processor, to execute any of the above method steps.

A multilevel cloud computing system that includes a central server, a plurality of local servers, and a plurality of vehicles provides distributed computation of data, allowing each level of the multilevel cloud computing system to improve data collection and processing. Using autonomous vehicles to detect erratic behavior of other erratic vehicles allows the central server to perform fleetwide actions while the local servers focus on localized actions. The local sensors can collect data with sensors mounted to infrastructure to identify erratically moving vehicles. The local servers can send messages to the central server identifying the erratically moving vehicles, allowing the central server to address the erratically moving vehicles on a fleetwide scale while the local servers and the vehicles perform additional computations to identify the erratically moving vehicles.

FIG. 1is a diagram of an example system100for detecting erratically moving vehicles. The system100includes a central server105. The central server105is a remote site that stores and transmits data. The central server105includes a processor and a memory, e.g., a data store. The central server105can include programming for managing a fleet of vehicles, as described below.

The system100includes one or more local servers110. The central server105can communicate with the local servers110. Each local server110includes a respective processor and memory. In this context, “local” means that the local servers110are disposed at specified locations from or at which the local server does not move (absent being uninstalled or the like), and that the local servers110each collect data from a predetermined area around the respective local server110, e.g., 400 square meters. For example, each local server110can be located at an intersection of two or more roadways. The local servers110can be located such that all locations in a specific geographic area (e.g., a city, a municipal county, etc.) can be detected by at least one local server110. Each local server110can communicate with other local servers110to exchange data, e.g., data about one or more vehicles115as described below.

The local servers110communicate with one or more vehicles115. Each vehicle115includes a computer including a processor and a memory. The computer is programmed to receive collected data from one or more sensors, e.g., vehicle115sensors, concerning various metrics related to the vehicle115. For example, the metrics may include a velocity of the vehicle115, vehicle115acceleration and/or deceleration, data related to vehicle115path or steering, biometric data related to a vehicle115operator, e.g., heart rate, respiration, pupil dilation, body temperature, state of consciousness, etc. Further examples of such metrics may include measurements of vehicle systems and components (e.g. a steering system, a powertrain system, a brake system, internal sensing, external sensing, etc.).

The computer is generally programmed for communications on a controller area network (CAN) bus or the like. The computer may also have a connection to an onboard diagnostics connector (OBD-II). Via the CAN bus, OBD-II, and/or other wired or wireless mechanisms, the computer may transmit messages to various devices in a vehicle and/or receive messages from the various devices, e.g., controllers, actuators, sensors, etc., including sensors. Alternatively or additionally, in cases where the computer actually comprises multiple devices, the CAN bus or the like may be used for communications between devices represented as the computer in this disclosure. In addition, the computer may be programmed for communicating with the network, which may include various wired and/or wireless networking technologies, e.g., cellular, Bluetooth, wired and/or wireless packet networks, etc. The computer may communicate with the local servers110over the network.

Collected data may include a variety of data collected in a vehicle115. Examples of collected data are provided above, and moreover, data is generally collected using one or more sensors, and may additionally include data calculated therefrom in the computer, and/or at the local server110and/or the central server105. In general, collected data may include any data that may be gathered by the sensors and/or computed from such data.

When the computer partially or fully operates the vehicle115, the vehicle115is an “autonomous” vehicle115. For purposes of this disclosure, the term “autonomous vehicle” is used to refer to a vehicle115operating in a fully autonomous mode. A fully autonomous mode is defined as one in which each of vehicle propulsion, braking, and steering are controlled by the computer. A semi-autonomous mode is one in which at least one of vehicle propulsion, braking, and steering are controlled at least partly by the computer as opposed to a human operator. In a non-autonomous mode, i.e., a manual mode, the vehicle propulsion, braking, and steering are controlled by the human operator.

The system100includes one or more sensors120. Sensors120can include a variety of devices, including cameras, motion detectors, etc., i.e., sensors120to provide data for evaluating a position of a vehicle115, evaluating a speed of a vehicle115, etc. The sensors120could, without limitation, also include short range radar, long range radar, lidar, and/or ultrasonic transducers. The sensors120can be mounted to infrastructure, e.g., a roadway sign, a roadway light such as a traffic light, a light pole, etc., at an intersection. The sensors120can collect data about one or more vehicles115and transmit the data to the local server110. That is, each local server110can communicate with one or more sensors120within a specific area (e.g., 400 square meters) to collect data about one or more vehicles115. The sensors120within the specific area can be “local” sensors120.

The central server105, the local servers110, the vehicles115, and the sensors120communicate over a network (not shown). The network represents one or more mechanisms by which a vehicle115computer may communicate with the local server110and the local server110can communicate with the central server105. Accordingly, the network can be one or more of various wired or wireless communication mechanisms, including any desired combination of wired (e.g., cable and fiber) and/or wireless (e.g., cellular, wireless, satellite, microwave, and radio frequency) communication mechanisms and any desired network topology (or topologies when multiple communication mechanisms are utilized). Exemplary communication networks include wireless communication networks (e.g., using Bluetooth®, Bluetooth® Low Energy (BLE), IEEE 802.11, a vehicle-to-everything network (V2X), where “X” signifies an entity with which a vehicle can communicate, e.g., vehicle-to-vehicle (V2V) such as Dedicated Short Range Communications (DSRC), etc.), vehicle-to-infrastructure (V2I), vehicle-to-network (V2N), vehicle-to-device (V2D), vehicle-to-grid (V2G), vehicle-to-pedestrian (V2P) such as a cellular-V2X (C-V2X) network, etc., local area networks (LAN) and/or wide area networks (WAN), including the Internet, providing data communication services.

The central server105, the local servers110, and the vehicles115are respective layers of a multilayer cloud computing system. In this context, a “multilayer cloud computing system” is a plurality of computers (e.g., servers, vehicle computers, etc.) communicating over a network, where each computer belongs to a “layer.” A “layer” is a set of computers that provides a certain kind of data and/or calculation and/or determination to other layers. Each layer includes one or more devices that collect and transmit data. That is, each device in each layer of the multilayer cloud computing system can perform specific actions and collect specific data that is transmitted to devices of the other layers. Computers in each layer transmit and receive data from computers in the other layers. In the example ofFIG. 1, the multilayer cloud computing system includes a “cloud” layer including the central server105, a “fog” layer including the local servers110, and an “edge” layer including the vehicles115. The edge layer collects and processes granular data and selects relevant data to send to the fog layer. The fog layer aggregates the selected data from the edge layer, processes the data, and selects data to send to the cloud layer, regulating data between the edge layer and the cloud layer. The cloud layer collects and processes the selected data from the fog layer to provide instructions to the fog layer and to the edge layer. By distributing the computing among multiple layers, the multilayer cloud computing system can process data closer in time to collection of the data, improving data collection and processing for operation of the vehicles115to identify erratically moving vehicles115and mitigate potential collisions. That is, the central server105can focus on fleetwide operations, e.g., monitoring a plurality of vehicles115, and the local servers110can identify erratically moving vehicles115without input from the central server105, reducing the computations required by the central server105. The fog and edge layers allow for processing of the data decentralized from the cloud layer, allowing the cloud layer to receive and manage processed data.

As described above, the central server105can be a “cloud layer,” i.e., a remote site that includes one or more computers and/or servers that collect and process data for management of a plurality of vehicles115over a large geographic area and stores data. The local servers110can be a “fog layer,” i.e., a localized computer focused on smaller geographic areas and fewer vehicles115. The local servers110can collect and process data that the central server105may not have capacity to process, e.g., predicting trajectories of vehicles115. The vehicles115can be an “edge layer” of the multilayer cloud computing system, i.e., individual vehicles115that include computers to collect and process real-time data for one or more nearby vehicles115. The vehicles115can process data that the local servers110may not have capacity to process, e.g., detecting and recording a speed of another vehicle115.

FIG. 2is a plan view of a local server110and a plurality of vehicles115at an intersection, i.e., a location where two or more roadways intersect. The local server110can be disposed on infrastructure, e.g., a traffic light200. The local server110can instruct one or more sensors120to collect data about the vehicles115. For example, the sensors120can collect steering, speed, and position data for each vehicle115, e.g., according to one or more various known techniques, e.g., by collecting images from cameras to determine positions of the vehicles115in a roadway, collecting radar data to determine speeds of the vehicles115, collecting lidar data to generate point clouds showing surfaces of the vehicles115over time to predict steering paths of the vehicles115, etc. In another example, the sensors120can collect image data of the vehicles115and roadway lane markings210, as described below, to determine positions of the vehicles115relative to the roadway lane markings210and whether the vehicles115are crossing over the roadway lane markings210. Each vehicle115can collect data about other nearby vehicles115and transmit the data to the local server110over the network. For example, each vehicle115can collect speed data of each other vehicle115.

The local server110identifies an erratically moving vehicle115. In this context, a vehicle moves “erratically” when movement of a vehicle115exceeds one of a plurality of predetermined standards for vehicle115operation. That is, “erratic” movement, or a vehicle behaving “erratically,” is movement of a subject vehicle115exceeding one or more thresholds specified for conventional (i.e., non-erratic) operation of a plurality of vehicles115. For example, the local server110can determine that a vehicle115is moving erratically when a speed of the vehicle115exceeds a speed threshold, e.g., a posted speed limit, an average speed of one or more vehicles115detected by the local server110, etc.

In another example, the local server110can determine that a vehicle115is moving erratically based on a position of the vehicle115in a roadway lane205relative to roadway lane marking210. As described above, the local server110can instruct one or more sensors120to collect data (e.g., image data, radar data, lidar data, etc.) of the vehicle115, the roadway lane205, and the roadway lane markings210. The local server110can use a conventional image-recognition technique, e.g., Canny edge detection, gradient matching, scale-invariant feature transforms, etc., to identify the vehicle115, the roadway lane205, and the roadway lane markings210. The local server110can determine the position of the vehicle115in the roadway lane205relative to the roadway lane markings210with a conventional distance determining technique, e.g., blur detection, a reference such as a known length of one of the roadway lane markings210, etc. If a vehicle115is swerving in the roadway lane205, the local server110and/or another vehicle115can determine that the vehicle115is moving erratically. In this context, the vehicle is “swerving” when the vehicle115moves toward a first roadway lane marking210and then moves toward a second roadway lane marking210laterally disposed from the first roadway lane marking210in an elapsed time that is below a time threshold. The time threshold can be a time to move from the first roadway lane marking210to the second roadway lane marking210at a speed beyond which the vehicle115will cross the second roadway lane marking210when the steering component120steers the vehicle115away from the second roadway lane marking210. The speed can be determined based on empirical testing and/or simulation testing of vehicles115moving toward roadway lane markings210at specific speeds and attempting to steer away from the roadway lane markings210. That is, each roadway lane205is defined by two sets of roadway lane markings210, and when the vehicle115quickly moves between the roadway lane markings, the vehicle115“swerves.”

In another example, the local server110can determine that a vehicle115is moving erratically when a steering angle of the vehicle115exceeds a steering angle threshold. The steering angle threshold can be a steering angle at which the vehicle115would leave the current roadway lane205in a time threshold (e.g., 3 seconds). That is, the steering angle threshold can be determined as a steering angle that would typically cause the vehicle115to leave the current roadway lane205, e.g., as determined based on empirical testing of vehicles115of specific steering angles between roadway lane markings210. The local server110can detect the steering angle of the vehicles115with the sensors120. For example, the local server110can instruct the sensors120to collect image data of the vehicles115over a period of time, e.g., 500 milliseconds (ms) in 50 ms increments, and can identify changes in positions of the vehicles115during the period of time with an image recognition technique as described above, Based on changes in the positions of the vehicles115in a vehicle-crosswise direction (i.e., transverse to forward motion of the vehicles115), the local server110can detect the change in steering angles of the vehicles115. Alternatively or additionally, steering angle sensors in the vehicles115can determine steering angles of the vehicles115and transmit the steering angle data over the network to the local server110.

In another example, the local server110can determine that a vehicle115is moving erratically when a position of the vehicle115is on a portion of a roadway where vehicles115are not permitted to operate. That is, portions of roadways can be designated as impermissible to travel, e.g., portions under construction, portions of a shoulder, portions past a stop sign without previously stopping, portions past a stoplight during a red light, etc. That is, the portions of the roadway can be designated as impermissible for travel based on local traffic regulations. The local server110can identify the position of the vehicle115based on, e.g., image data from one or more sensors120as described above. The local server110can compare the identified position of the vehicle115to stored locations that are identified as unpermitted for vehicle115operation. The local server110can receive the locations identified as unpermitted from the central server105. When the local server110identifies the location of the vehicle115as a portion of the roadway where vehicles115are not permitted to operate, the local server110can identify the vehicle115as an erratically moving vehicle115. In another example, if the position data of the vehicle115indicate that the vehicle115is partly disposed in two roadway lanes (i.e., the vehicle115is moving over a roadway lane marking210but not changing roadway lanes205), the local server110can identify the vehicle115as an erratically moving vehicle115.

The local server110can determine the threshold(s) for erratic behavior such as described above based on data collected from the vehicles115and/or one or more sensors120. The data can include speed data, location data, movement data, etc. The local server110can input the data into a machine learning program to identify thresholds such as the speed threshold and the time threshold described above. The machine learning program can be, e.g. a convolutional neural network. The machine learning program can be trained by inputting steering, speed, and position data from reference vehicles115that are not identified as moving erratically and adjusting coefficients of a cost function using, e.g., gradient descent, to output that the reference vehicles115are not moving erratically. With the steering, speed, and position data, the machine learning program can be trained to identify respective ranges for steering angle, speed, and position of the vehicles115and an average steering angle, speed, and position. The machine learning program can output the ranges for the steering angle, speed, and position that the local servers110can use as respective thresholds for the steering angle, speed, and position. For example, if input speed data indicate that a speed range of the vehicles is 40-50 miles per hour (mph), the machine learning program can output the average speed as 45 mph and the speed range as 5 mph. Then, upon detecting a speed of a vehicle115that is outside the speed range from the average speed, e.g., less than 40 mph or more than 50 mph, the local server110can identify the vehicle115as an erratically moving vehicle115. In another example, input steering data to the machine learning program can result in an output that a steering angle range is −5°-5° and an average steering angle of 0°. The local server110can use these outputs as respective thresholds for determining whether a vehicle115is moving erratically.

Upon identifying an erratically moving vehicle115, the local server110can instruct the erratically moving vehicle115to move out of a current roadway lane205to a side of the roadway. A “side” of the roadway is a portion of the roadway designated for vehicles115to stop, e.g., a shoulder. The local server110can store a set of geo-coordinate data that are locations of sides of roadways, and the local server110can instruct the erratically moving vehicle115to move to specific geo-coordinates of the side of the roadway. By instructing the erratically moving vehicle115to move to the side of the roadway, the local server110removes the erratically moving vehicle115from the roadway lane205and from other vehicles115, reducing the likelihood of the erratically moving vehicle115colliding with another vehicle115.

Upon instructing the erratically moving vehicle115to move to the side of the roadway, the local server110can send a message to the central server105identifying the erratically moving vehicle115. The message can include a location of the erratically moving vehicle115and/or an indication of the criteria that the local server110used to identify the erratically moving vehicle115. For example, the message can include the speed of the erratically moving vehicle115that exceeded the speed threshold.

Upon receiving the message indicating the erratically moving vehicle115, the central server105can instruct the vehicle115to mitigate the erratic movement. For example, the central server105can instruct the vehicle115to move to a repair location to repair one or more components that may be fault and may have caused the erratic movement. In another example, the central server105can instruct the vehicle115to receive data from another vehicle115and/or the local server110and to use the data to operate one or more components. The central server105can instruct the erratically moving vehicle115to use data from the local server110and/or other vehicles115that may be more reliable than data collected by the erratically moving vehicle115. That is, the erratically moving vehicle115may have one or more sensors that are faulty, i.e., damaged and/or incorrectly collecting data, and using data from the local server110and/or the other vehicles115may mitigate the erratic movement of the erratically moving vehicle115. In yet another example, the central server105can send a human operator to the erratically moving vehicle115to diagnose one or more faults that caused the erratic movement and to operate the vehicle115in a manual mode to a repair location.

FIG. 4is a diagram of an example process400for detecting erratically moving vehicles115. The process400begins in a block405, in which a local server110collects data about a plurality of vehicles115. The local sever110can collect the data from one or more sensors120mounted to infrastructure at an intersection. For example, the local server110can collect data from a plurality of cameras120mounted to a traffic light200.

Next, in a block410, the local server110collects data from a plurality of vehicles115about other vehicles115at the intersection. Each vehicle115can collect data such as image data, speed data, steering data, etc., about other vehicles115and can transmit the data to the local server110. That is, each vehicle115includes a plurality of sensors that collect data, and each vehicle115can transmit the data to the local server110.

Next, in a block415, the local server110determines whether data about one of the plurality of vehicles115indicate that the vehicle115exceeds a speed threshold or a position threshold. As described above, when a speed of the vehicle115exceeds a speed threshold or a position of the vehicle115indicates that the vehicle115swerves or is on a portion of a roadway unpermitted for vehicles115, the local server110can determine that the vehicle115is behaving erratically. As described above, the speed threshold and the position threshold can be determined based on, e.g., posted speed limits, average speed and position data of other vehicles115, location map data indicating unpermitted areas for vehicles115, etc. If the local server110determines that the data indicate that one of the vehicles115exceeds the speed threshold or the position threshold, the process400continues in a block420. Otherwise, the process400continues in a block430.

In the block420, the local server110instructs the erratically moving vehicle115to move to a side of the roadway. As described above, when the erratically moving vehicle115moves to the side of the roadway and stops, the vehicle115is away from other vehicles115in the roadway, reducing a likelihood of a collision between the erratically moving vehicle115and other vehicles115.

Next, in a block425, the local server110sends a message to the central server105identifying the erratically moving vehicle115. As described above, the message can include a location of the vehicle115, the data that indicated to the local server110that the vehicle115was moving erratically, etc. The central server105can instruct the vehicle115to perform countermeasures to address the erratic movement, e.g., the central server105can instruct the vehicle115to move to a repair location to repair one or more components that could have cause the erratic movement of the vehicle115.

In the block430, the local server110determines whether to continue the process400. For example, the local server110can determine to continue the process400upon detecting additional vehicles115at the intersection. If the local server110determines to continue, the process400returns to the block405to collect additional data. Otherwise, the process400ends.

Computing devices discussed herein, including the central server105and the local server110include processors and memories, the memories generally each including instructions executable by one or more computing devices such as those identified above, and for carrying out blocks or steps of processes described above. Computer executable instructions may be compiled or interpreted from computer programs created using a variety of programming languages and/or technologies, including, without limitation, and either alone or in combination, Java™, C, C++, Python, Visual Basic, Java Script, Perl, HTML, etc. In general, a processor (e.g., a microprocessor) receives instructions, e.g., from a memory, a computer readable medium, etc., and executes these instructions, thereby performing one or more processes, including one or more of the processes described herein. Such instructions and other data may be stored and transmitted using a variety of computer readable media. A file in the local server110is generally a collection of data stored on a computer readable medium, such as a storage medium, a random access memory, etc.

With regard to the media, processes, systems, methods, etc. described herein, it should be understood that, although the steps of such processes, etc. have been described as occurring according to a certain ordered sequence, such processes could be practiced with the described steps performed in an order other than the order described herein. It further should be understood that certain steps could be performed simultaneously, that other steps could be added, or that certain steps described herein could be omitted. For example, in the process400, one or more of the steps could be omitted, or the steps could be executed in a different order than shown inFIG. 4. In other words, the descriptions of systems and/or processes herein are provided for the purpose of illustrating certain embodiments, and should in no way be construed so as to limit the disclosed subject matter.

The article “a” modifying a noun should be understood as meaning one or more unless stated otherwise, or context requires otherwise. The phrase “based on” encompasses being partly or entirely based on.