Patent ID: 12248321

DETAILED DESCRIPTION

In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the various embodiments. It will be evident, however, to one of ordinary skill in the art that the various embodiments may be practiced without these specific details.

A system and method for generating simulated vehicles with configured behaviors for analyzing autonomous vehicle motion planners is disclosed herein. Specifically, the present disclosure describes a dynamic vehicle simulation system to generate simulated dynamic vehicles with various driving behaviors to test, evaluate, or otherwise analyze autonomous vehicle motion planning systems, which will be used in real autonomous vehicles in actual driving environments. The simulated dynamic vehicles (also denoted herein as non-player characters or NPC vehicles) generated by the simulation system of various example embodiments described herein can model the vehicle behaviors that would be performed by actual vehicles in the real world, including lane change, overtaking, acceleration behaviors, and the like.

Referring toFIG.1, a diagram illustrates the components of an example embodiment of the dynamic vehicle simulation system102and the dynamic vehicle simulation module200therein. In particular, the dynamic vehicle simulation system102can include a perception data collection module204. The perception data collection module204can be executed by a data processor171of the dynamic vehicle simulation system102. The perception data collection module204can include an array of interfaces for receiving perception data from a plurality of perception information sensing devices or perception data sensors110. The perception data sensors110may include image generating devices (e.g., cameras), light amplification by stimulated emission of radiation (laser) devices, light detection and ranging (LIDAR) devices, global positioning system (GPS) devices, sound navigation and ranging (sonar) devices, radio detection and ranging (radar) devices, other distance measuring systems, and the like. The perception data gathered by the perception data sensors110at various traffic locations can include traffic or vehicle image data, roadway data, environmental data, distance data from LIDAR or radar devices, and other sensor information received from the perception data sensors110positioned adjacent to particular roadways (e.g., monitored locations) or installed on stationary test vehicles. Additionally, the perception data sensors110can include perception data gathering devices installed in or on moving test vehicles being navigated through pre-defined routings in an environment or location of interest. The perception data can include data from which a presence, position, and velocity of neighboring vehicles in the vicinity of or proximate to a host vehicle, autonomous vehicle, or simulated vehicle can be obtained or calculated.

The perception data collection module204can collect actual trajectories of vehicles under different scenarios and different driver behaviors. The different scenarios can correspond to different locations, different traffic patterns, different environmental conditions, and the like. The scenarios can be represented, for example, by an occupancy grid, a collection of vehicle states on a map, or a graphical representation, such as a top-down image of one or more areas of interest. The driver behaviors can correspond to a driver's short term driving activity, such as changing lanes to the left or right, overtaking other vehicles, accelerating/decelerating, merging to/from a ramp, making left or right turn at an intersection, making a U-turn, and the like. The driver behaviors can also correspond to a set of driver or vehicle control actions to accomplish the particular short term driving activity.

The image data and other perception data collected by the perception data collection module204reflects truly realistic, real-world traffic environment information related to the locations or routings, the scenarios, and the driver behaviors being monitored. Using the standard capabilities of well-known data collection devices, the gathered traffic and vehicle image data and other perception or sensor data can be wirelessly transferred (or otherwise transferred) to a data processor of a standard computing system, upon which the perception data collection module204can be executed. Alternatively, the gathered traffic and vehicle image data and other perception or sensor data can be stored in a memory device at the monitored location or in the test vehicle and transferred later to the data processor of the standard computing system. The traffic and vehicle image data and other perception or sensor data, and the driver behavior data gathered or calculated by the perception data collection module204can be used to generate simulated proximate dynamic vehicles for a simulation environment as described in more detail below.

Referring againFIG.1, additional components of the dynamic vehicle simulation system102are illustrated. As described above, the dynamic vehicle simulation system102can gather the perception data collected by the perception data collection module204. This perception data can be used in a simulation environment, produced by the dynamic vehicle simulation system102, to create corresponding simulations of proximate dynamic vehicles or object trajectories in the simulation environment. As a result, the example embodiments use the perception data collection module204to collect perception data that can be used to infer corresponding human driving behaviors. Then, the example embodiments can use the dynamic vehicle simulation system102in the simulation environment to simulate proximate dynamic vehicles with configurable human driving behaviors based in part on the collected perception data.

Referring stillFIG.1, the dynamic vehicle simulation system102can include a dynamic vehicle configuration module206, a set of dynamic vehicle configuration data208, and a dynamic vehicle simulation module210. The dynamic vehicle configuration module206and the dynamic vehicle simulation module210can be executed by a data processor171of the dynamic vehicle simulation system102. The dynamic vehicle configuration data208can be stored in a memory device or system172of the dynamic vehicle simulation system102. The dynamic vehicle configuration module206can be configured to read portions of the pre-defined data retained as the dynamic vehicle configuration data208to obtain pre-defined parameters and executables for each of a plurality of dynamic vehicles being simulated by the dynamic vehicle simulation module210, described in more detail below. The pre-defined parameters and executables for each simulated dynamic vehicle constitute configuration instructions and data defining a specific driving behavior for each of a plurality of dynamic vehicles being simulated. The configuration instructions and data enable the dynamic vehicle simulation module210to generate a simulation of a particular dynamic vehicle with a specific driving behavior. For example, the configuration instructions and data for a particular dynamic vehicle can cause the dynamic vehicle simulation module210to generate a simulation of the particular dynamic vehicle with an aggressive driving behavior. In the example, the aggressive driving behavior can correspond to a simulated dynamic vehicle that frequently changes lanes, exhibits steep acceleration and deceleration rates, and travels close to other neighboring vehicles. In contrast, the configuration instructions and data for a particular dynamic vehicle can cause the dynamic vehicle simulation module210to generate a simulation of the particular dynamic vehicle with a conservative driving behavior. In this example, the conservative driving behavior can correspond to a simulated dynamic vehicle that infrequently changes lanes, exhibits moderate acceleration and deceleration rates, and maintains a greater distance from other neighboring vehicles. It will be apparent to those of ordinary skill in the art in view of the disclosure herein that a variety of other specific driving behaviors can be simulated using the configuration instructions and data defined in the dynamic vehicle configuration data208and processed by the dynamic vehicle configuration module206.

The dynamic vehicle simulation module210, of an example embodiment as shown inFIG.1, can receive the perception data from the perception data collection module204and the configuration instructions and data for each dynamic vehicle to be simulated. The received perception data can inform the dynamic vehicle simulation module210of the environment surrounding the particular dynamic vehicle being simulated. For example, the perception data can include information indicative of the presence, position, and velocity of neighboring vehicles in the vicinity of or proximate to a host vehicle, an autonomous vehicle, or simulated dynamic vehicle. The perception data can also include information indicative of the presence and position of obstacles, the location of the available roadways, and other environmental information. The configuration instructions and data for each dynamic vehicle to be simulated can inform the dynamic vehicle simulation module210of the specific configurable driving behaviors to be modeled for the particular dynamic vehicle being simulated. Given the perception data and the configuration instructions and data for each dynamic vehicle being simulated, the dynamic vehicle simulation module210can generate a proposed or target position, speed, and heading for each particular dynamic vehicle being simulated at specific points in time. The proposed or target position, speed, and heading for each simulated dynamic vehicle can be generated based on the received perception data and the specific configuration instructions and data. In an example embodiment, the dynamic vehicle simulation module210can use a rule-based process and corresponding data structures to determine and generate the target position, speed, and heading corresponding to the specific behavior of each simulated dynamic vehicle based on the configuration instructions and data corresponding to each simulated dynamic vehicle. In the example embodiment, the specific behavior of a simulated dynamic vehicle, as represented in the rule-based process and corresponding data structures, can be modeled using the target position and direction of the simulated dynamic vehicle along with the target speed of the simulated dynamic vehicle. The target position/direction and target speed of the simulated dynamic vehicle correspond to the position and speed of a vehicle that would be expected given the pre-defined dynamic vehicle configuration data208as described above. Thus, the target position/direction and target speed of the simulated dynamic vehicle correspond to the pre-configured behavior for a specific simulated dynamic vehicle. Because the target position/direction and target speed of the simulated dynamic vehicle correspond to the pre-configured behavior, the target position/direction and target speed of each simulated dynamic vehicle is likely to be different. For example, a particular simulated dynamic vehicle having a pre-configured behavior corresponding to an aggressive driver may be more likely (higher probability) to have a determined target position/direction and target speed associated with a lane change, passing maneuver, sharp turn, or sudden stop. A different simulated dynamic vehicle having a pre-configured behavior corresponding to a conservative driver may be less likely (lower probability) to have a determined target position/direction and target speed associated with a lane change, passing maneuver, sharp turn, or sudden stop. As a result, a target position/direction and target speed of each simulated dynamic vehicle conforming to the configured driving behavior for each simulated dynamic vehicle can be generated by the dynamic vehicle simulation module210. The target position/direction and target speed can be passed to a trajectory generator212as shown inFIG.1.

As illustrated inFIG.1, the trajectory generator212can receive the target position/direction and target speed generated by the dynamic vehicle simulation module210for each simulated dynamic vehicle as described above. The trajectory generator212can generate a trajectory to transition a particular simulated dynamic vehicle from its current position/direction and speed to the target position/direction and target speed as generated by the dynamic vehicle simulation module210. In an example embodiment, the trajectory generator212can include a path sampler module214and a speed sampler module216. The path sampler module214can generate multiple paths or trajectories from the particular simulated dynamic vehicle's current position to the target position. The multiple trajectories enable a selection of a particular trajectory based on the presence of obstacles or the accommodation of other simulation goals, such as safety, fuel-efficiency, and the like. The speed sampler module216can generate multiple acceleration profiles to transition the particular simulated dynamic vehicle from its current speed to the target speed. Again, the multiple acceleration profiles enable a selection of a particular acceleration profile based on the presence of obstacles or the accommodation of other simulation goals, such as safety, fuel-efficiency, and the like. The multiple trajectories and multiple acceleration profiles for each simulated dynamic vehicle can be represented as waypoints each having a corresponding position, speed, acceleration, and time. The waypoints generated by the trajectory generator212can represent the movements and behaviors of each simulated dynamic vehicle in the simulation environment. As shown inFIG.1, the trajectory corresponding to each of a plurality of simulated dynamic vehicles can be provided as an output from the dynamic vehicle simulation system102and an input to an autonomous vehicle controller182. The autonomous vehicle controller182can include a motion planner module used to generate a trajectory for an autonomous vehicle based on the environment around the autonomous vehicle and the destination or goals of the autonomous vehicle. The environment around the autonomous vehicle can include the presence, position, heading, and speed of proximate vehicles or other objects near the autonomous vehicle. Given the trajectories corresponding to a plurality of simulated dynamic vehicles as provided by the dynamic vehicle simulation system102, the motion planner module in the autonomous vehicle controller182can be stimulated to react to the presence and behavior of the simulated dynamic vehicles just as the motion planner would react to the presence and behavior of real vehicles in a real world driving environment. In this manner, the dynamic vehicle simulation system102can be used to produce trajectories corresponding to a plurality of simulated dynamic vehicles, which can be used to stimulate the motion planner of an autonomous vehicle. The trajectories produced by the motion planner in response to the plurality of simulated dynamic vehicles can be analyzed to determine if the motion planner is producing acceptable output. As described above, the behaviors of the simulated dynamic vehicles generated by the dynamic vehicle simulation system102can be configured, modified, and specifically tuned to produce a wide range of driving behaviors, environments, scenarios, and tests to exercise the full capabilities of the autonomous vehicle motion planner. As a result of the processing performed by the dynamic vehicle simulation system102as described above, data corresponding to simulated driver and vehicle behaviors and corresponding simulated dynamic vehicle trajectories can be produced. Ultimately, the dynamic vehicle simulation system102can be used to provide highly configurable simulated traffic trajectory information to a user or for configuration or analysis of a control system of an autonomous vehicle. In particular, the simulated traffic trajectory information can be used to create a virtual world where a control system for an autonomous vehicle can be analyzed, modified, and improved. The virtual world is configured to be identical (as possible) to the real world where vehicles are operated by human drivers. In other words, the simulated traffic trajectory information generated by the dynamic vehicle simulation system102is highly useful for configuring and analyzing the control systems of an autonomous vehicle. It will be apparent to those of ordinary skill in the art that the dynamic vehicle simulation system102and the simulated traffic trajectory information described and claimed herein can be implemented, configured, processed, and used in a variety of other applications and systems as well.

Referring again toFIG.1, the dynamic vehicle simulation system102can be configured to include executable modules developed for execution by a data processor171in a computing environment of the dynamic vehicle simulation system102and the dynamic vehicle simulation module200therein. In the example embodiment, the dynamic vehicle simulation module200can be configured to include the plurality of executable modules as described above. A data storage device or memory172can also be provided in the dynamic vehicle simulation system102of an example embodiment. The memory172can be implemented with standard data storage devices (e.g., flash memory, DRAM, SIM cards, or the like) or as cloud storage in a networked server. In an example embodiment, the memory172can be used to store the set of dynamic vehicle configuration data208as described above. In various example embodiments, the set of dynamic vehicle configuration data208can be configured to simulate more than the typical driving behaviors. To simulate an environment that is identical to the real world as much as possible, the dynamic vehicle configuration data208can represent typical driving behaviors, which represent average drivers. Additionally, the dynamic vehicle configuration data208can also represent atypical driving behaviors. In most cases, the trajectories corresponding to the plurality of simulated dynamic vehicles include typical and atypical driving behaviors. As a result, autonomous vehicle motion planners can be stimulated by the dynamic vehicle simulation system102using trajectories related to the driving behaviors of polite and impolite drivers as well as patient and impatient drivers in the virtual world. In all, the simulated dynamic vehicles can be configured with data representing driving behaviors that are as varied as possible.

Referring now toFIG.2, a flow diagram illustrates an example embodiment of a system and method1000for dynamic vehicle simulation. The example embodiment can be configured for: receiving perception data from a plurality of perception data sensors (processing block1010); obtaining configuration instructions and data including pre-defined parameters and executables defining a specific driving behavior for each of a plurality of simulated dynamic vehicles (processing block1020); generating a target position and target speed for each of the plurality of simulated dynamic vehicles, the generated target positions and target speeds being based on the perception data and the configuration instructions and data (processing block1030); and generating a plurality of trajectories and acceleration profiles to transition each of the plurality of simulated dynamic vehicles from a current position and speed to the corresponding target position and target speed (processing block1040).

FIG.3shows a diagrammatic representation of a machine in the example form of a computing system700within which a set of instructions when executed and/or processing logic when activated may cause the machine to perform any one or more of the methodologies described and/or claimed herein. In alternative embodiments, the machine operates as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine may operate in the capacity of a server or a client machine in server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine may be a personal computer (PC), a laptop computer, a tablet computing system, a Personal Digital Assistant (PDA), a cellular telephone, a smartphone, a web appliance, a set-top box (STB), a network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) or activating processing logic that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term “machine” can also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions or processing logic to perform any one or more of the methodologies described and/or claimed herein.

The example computing system700can include a data processor702(e.g., a System-on-a-Chip (SoC), general processing core, graphics core, and optionally other processing logic) and a memory704, which can communicate with each other via a bus or other data transfer system706. The mobile computing and/or communication system700may further include various input/output (I/O) devices and/or interfaces710, such as a touchscreen display, an audio jack, a voice interface, and optionally a network interface712. In an example embodiment, the network interface712can include one or more radio transceivers configured for compatibility with any one or more standard wireless and/or cellular protocols or access technologies (e.g., 2nd (2G), 2.5, 3rd (3G), 4th (4G) generation, and future generation radio access for cellular systems, Global System for Mobile communication (GSM), General Packet Radio Services (GPRS), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), LTE, CDMA2000, WLAN, Wireless Router (WR) mesh, and the like). Network interface712may also be configured for use with various other wired and/or wireless communication protocols, including TCP/IP, UDP, SIP, SMS, RTP, WAP, CDMA, TDMA, UMTS, UWB, WiFi, WiMax, Bluetooth™, IEEE 802.11x, and the like. In essence, network interface712may include or support virtually any wired and/or wireless communication and data processing mechanisms by which information/data may travel between a computing system700and another computing or communication system via network714.

The memory704can represent a machine-readable medium on which is stored one or more sets of instructions, software, firmware, or other processing logic (e.g., logic708) embodying any one or more of the methodologies or functions described and/or claimed herein. The logic708, or a portion thereof, may also reside, completely or at least partially within the processor702during execution thereof by the mobile computing and/or communication system700. As such, the memory704and the processor702may also constitute machine-readable media. The logic708, or a portion thereof, may also be configured as processing logic or logic, at least a portion of which is partially implemented in hardware. The logic708, or a portion thereof, may further be transmitted or received over a network714via the network interface712. While the machine-readable medium of an example embodiment can be a single medium, the term “machine-readable medium” should be taken to include a single non-transitory medium or multiple non-transitory media (e.g., a centralized or distributed database, and/or associated caches and computing systems) that store the one or more sets of instructions. The term “machine-readable medium” can also be taken to include any non-transitory medium that is capable of storing, encoding or carrying a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the various embodiments, or that is capable of storing, encoding or carrying data structures utilized by or associated with such a set of instructions. The term “machine-readable medium” can accordingly be taken to include, but not be limited to, solid-state memories, optical media, and magnetic media.

The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment.