Patent Description:
Document <CIT> discloses, in the field of autonomous vehicle technology, automated training data set generators that generate feature training data sets for use in real-world autonomous driving applications based on virtual environments are disclosed herein. The feature training data sets are associated with training a machine learning model to control real-world autonomous vehicles. It includes an autonomous vehicle simulator that receives, as input, simulated sensor data from and environment-object data from a physical component, wherein the environment-object data defines how objects or surfaces interact with each other in the virtual environment. The training dataset generator operates entirely at the virtual environment level rather than with the logged data of the autonomous vehicle.

Document <CIT> discloses an autonomous vehicle management system can evaluate and simulate various "what-if ' scenarios. These what-if scenarios project various behavioural predictions onto an internal map and can be used to determine a safe sequence of actions to be performed by the autonomous vehicle in order to accomplish a particular goal.

This specification relates to methods and systems for generating simulation data from logged data. According to one aspect of the subject matter described in this disclosure, a method includes receiving logged data of the autonomous vehicle, generating augmented data from the logged data, the augmented data describing an actor in an environment of the autonomous vehicle, the actor having an associated actor type and an actor motion behavior characteristic, and generating a simulation scenario as the simulation data, the simulation scenario generated from the augmented data.

The invention is defined in independent claims <NUM> and <NUM>.

Further preferred embodiments of the invention are defined the dependent claims.

In general, another aspect of the subject matter described in this disclosure includes a system comprising one or more processors and memory operably coupled with the one or more processors, wherein the memory stores instructions that, in response to the execution of the instructions by one or more processors, cause the one or more processors to perform the following operations of receiving logged data of the autonomous vehicle, generating augmented data from the logged data, the augmented data describing an actor in an environment of the autonomous vehicle, the actor having an associated actor type and an actor motion behavior characteristic, and generating a simulation scenario as the simulation data, the simulation scenario generated from the augmented data.

Other implementations of one or more of these aspects include corresponding systems, apparatus, and computer programs, configured to perform the actions of the methods, encoded on computer storage devices.

These and other implementations include one or more of the following features. For instance, the method further comprises executing a simulation based on the simulation scenario to generate a simulated output, providing the simulation scenario as a training input to the machine learning model to generate a predicted output of the machine learning model, and updating one or more weights in the machine learning model based on a difference between the predicted output and the simulated output of the simulation scenario. For instance, features may also include that the logged data includes raw sensor data and one of data from a video game and data from film, or the logged data is time-series logged data including localization data and tracking data, or the logged data includes one of raw sensor data from any one or more sensors, state or localization data from a localization subsystem, state or perception data from perception subsystem, state or planning data from the planning subsystem and state or control data from the control subsystem. For example, the method may also include mapping the logged data to a coordinate system to produce mapped logged data, performing smoothing of the mapped logged data to produced smoothed data, and wherein the smoothed data is used in generating augmented data from the logged data. In general, other aspects of the subject matter of this disclosure may be implemented in methods where the simulation data comprises a simulation scenario that describes motion behavior of a simulated autonomous vehicle and at least one simulated actor. For instance, the method may also include identifying, from the logged data, actors and generating actor states to create an initial augmented data, sampling the initial augmented data; and generating a variation of the sampled augmented data. For example, features may also include that the generating of the variation includes changing one from a group of actor velocity, actor type, actor size, actor path, lateral offset of motion, longitudinal offset of motion, adding an actor, deleting an actor and actor behavior response. Still other implementations include generating a plurality of sets of sampled augmented data, and generating the simulation scenario from the augmented data includes generating a plurality of simulation scenarios each corresponding to one of the sets of sampled augmented data.

These and other aspects and features of the present implementations will become apparent upon review of the following description of specific implementations in conjunction with the accompanying figures, wherein:.

Implementations of the disclosure are generally related to the use of logged or real sensor data from autonomous vehicles to generate simulation data, such as simulation scenarios. The simulation scenarios can be in turn used to train machine learning models that are used in various subsystems of an autonomous vehicle, for example, the perception, planning, and control subsystems.

Referring to the drawings, wherein like numbers denote like parts throughout the several views, <FIG> illustrates an example hardware and software environment for an autonomous vehicle within which various techniques disclosed herein may be implemented. The vehicle <NUM>, for example, may include a powertrain <NUM> including a prime mover <NUM> powered by an energy source <NUM> and capable of providing power to a drivetrain <NUM>, as well as a control system <NUM> including a direction control <NUM>, a powertrain control <NUM>, and a brake control <NUM>. The vehicle <NUM> may be implemented as any number of different types of vehicles, including vehicles capable of transporting people and/or cargo, and capable of traveling by land, by sea, by air, underground, undersea, and/or in space, and it will be appreciated that the aforementioned components <NUM>-<NUM> may vary widely based upon the type of vehicle within which these components are utilized.

For simplicity, the implementations discussed hereinafter will focus on a wheeled land vehicle such as a car, van, truck, bus, etc. In such implementations, the prime mover <NUM> may include one or more electric motors and/or an internal combustion engine (among others). The energy source <NUM> may include, for example, a fuel system (e.g., providing gasoline, diesel, hydrogen, etc.), a battery system, solar panels or other renewable energy source, and/or a fuel cell system. The drivetrain <NUM> includes wheels and/or tires along with a transmission and/or any other mechanical drive components suitable for converting the output of the prime mover <NUM> into vehicular motion, as well as one or more brakes configured to controllably stop or slow the vehicle <NUM> and direction or steering components suitable for controlling the trajectory of the vehicle <NUM> (e.g., a rack and pinion steering linkage enabling one or more wheels of the vehicle <NUM> to pivot about a generally vertical axis to vary an angle of the rotational planes of the wheels relative to the longitudinal axis of the vehicle). In some implementations, combinations of powertrains and energy sources may be used (e.g., in the case of electric/gas hybrid vehicles), and in other implementations multiple electric motors (e.g., dedicated to individual wheels or axles) may be used as a prime mover. In the case of a hydrogen fuel cell implementation, the prime mover <NUM> may include one or more electric motors and the energy source <NUM> may include a fuel cell system powered by hydrogen fuel.

The direction control <NUM> may include one or more actuators and/or sensors for controlling and receiving feedback from the direction or steering components to enable the vehicle <NUM> to follow a desired trajectory. The powertrain control <NUM> may be configured to control the output of the powertrain <NUM>, e.g., to control the output power of the prime mover <NUM>, to control a gear of a transmission in the drivetrain <NUM>, etc., thereby controlling a speed and/or direction of the vehicle <NUM>. The brake control <NUM> may be configured to control one or more brakes that slow or stop vehicle <NUM>, e.g., disk or drum brakes coupled to the wheels of the vehicle.

Other vehicle types, including but not limited to all-terrain or tracked vehicles or construction equipment, will necessarily utilize different powertrains, drivetrains, energy sources, direction controls, powertrain controls and brake controls. Moreover, in some implementations, some of the components can be combined, e.g., where directional control of a vehicle is primarily handled by varying an output of one or more prime movers. Therefore, implementations disclosed herein are not limited to the particular application of the herein-described techniques in an autonomous wheeled land vehicle.

In the illustrated implementation, full or semi-autonomous control over the vehicle <NUM> is implemented in a vehicle control system <NUM>, which may include one or more processors <NUM> and one or more memories <NUM>, with each processor <NUM> configured to execute program code instructions <NUM> stored in a memory <NUM>. The processors(s) can include, for example, graphics processing unit(s) ("GPU(s)")) and/or central processing unit(s) ("CPU(s)").

Sensors <NUM> may include various sensors suitable for collecting information from a vehicle's surrounding environment for use in controlling the operation of the vehicle <NUM>. For example, sensors <NUM> can include RADAR sensor <NUM>, LIDAR (Light Detection and Ranging) sensor <NUM>, a 3D positioning sensor <NUM>, e.g., a satellite navigation system such as GPS (Global Positioning System), GLONASS (Globalnaya Navigazionnaya Sputnikovaya Sistema, or Global Navigation Satellite System), BeiDou Navigation Satellite System (BDS), Galileo, Compass, etc. The 3D positioning sensors <NUM> can be used to determine the location of the vehicle on the Earth using satellite signals. The sensors <NUM> can optionally include a camera <NUM> and/or an IMU (inertial measurement unit) <NUM>. The camera <NUM> can be a monographic or stereographic camera and can record still and/or video images. The IMU <NUM> can include multiple gyroscopes and accelerometers capable of detecting linear and rotational motion of the vehicle <NUM> in three directions. One or more encoders <NUM>, such as wheel encoders may be used to monitor the rotation of one or more wheels of vehicle <NUM>.

The outputs of sensors <NUM> may be provided to a set of control subsystems <NUM>, including, a localization subsystem <NUM>, a perception subsystem <NUM>, a planning subsystem <NUM>, and a control subsystem <NUM>. The localization subsystem <NUM> is principally responsible for precisely determining the location and orientation (also sometimes referred to as "pose") of the vehicle <NUM> within its surrounding environment, and generally within some frame of reference. The perception subsystem <NUM> is principally responsible for detecting, tracking, and/or identifying objects within the environment surrounding vehicle <NUM>. A machine learning model in accordance with some implementations can be utilized in tracking objects. The planning subsystem <NUM> is principally responsible for planning a trajectory or a path of motion for vehicle <NUM> over some timeframe given a desired destination as well as the static and moving objects within the environment. A machine learning model in accordance with some implementations can be utilized in planning a vehicle trajectory. The control subsystem <NUM> is principally responsible for generating suitable control signals for controlling the various controls in the vehicle control system <NUM> in order to implement the planned trajectory of the vehicle <NUM>. Similarly, a machine learning model can be utilized to generate one or more signals to control the autonomous vehicle <NUM> to implement the planned trajectory.

It will be appreciated that the collection of components illustrated in <FIG> for the vehicle control system <NUM> is merely one example. Individual sensors may be omitted in some implementations. Additionally, or alternatively, in some implementations, multiple sensors of the same types illustrated in <FIG> may be used for redundancy and/or to cover different regions around a vehicle. Moreover, there may additional sensors of other types beyond those described above to provide actual sensor data related to the operation and environment of the wheeled land vehicle. Likewise, different types and/or combinations of control subsystems may be used in other implementations. Further, while subsystems <NUM>-<NUM> are illustrated as being separate from processor <NUM> and memory <NUM>, it will be appreciated that in some implementations, some or all of the functionality of a subsystem <NUM>-<NUM> may be implemented with program code instructions <NUM> resident in one or more memories <NUM> and executed by one or more processors <NUM>, and that these subsystems <NUM>-<NUM> may in some instances be implemented using the same processor(s) and/or memory. Subsystems may be implemented at least in part using various dedicated circuit logic, various processors, various field programmable gate arrays ("FPGA"), various application-specific integrated circuits ("ASIC"), various real time controllers, and the like, as noted above, multiple subsystems may utilize circuitry, processors, sensors, and/or other components. Further, the various components in the vehicle control system <NUM> may be networked in various manners.

In some implementations, the vehicle <NUM> may also include a secondary vehicle control system (not illustrated), which may be used as a redundant or backup control system for the vehicle <NUM>. In some implementations, the secondary vehicle control system may be capable of fully operating the autonomous vehicle <NUM> in the event of an adverse event in the vehicle control system <NUM>, while in other implementations, the secondary vehicle control system may only have limited functionality, e.g., to perform a controlled stop of the vehicle <NUM> in response to an adverse event detected in the primary vehicle control system <NUM>. In still other implementations, the secondary vehicle control system may be omitted.

In general, an innumerable number of different architectures, including various combinations of software, hardware, circuit logic, sensors, networks, etc. may be used to implement the various components illustrated in <FIG>. Each processor may be implemented, for example, as a microprocessor and each memory may represent the random access memory ("RAM") devices comprising a main storage, as well as any supplemental levels of memory, e.g., cache memories, non-volatile or backup memories (e.g., programmable or flash memories), read-only memories, etc. In addition, each memory may be considered to include memory storage physically located elsewhere in the vehicle <NUM>, e.g., any cache memory in a processor, as well as any storage capacity used as a virtual memory, e.g., as stored on a mass storage device or another computer controller. One or more processors <NUM> illustrated in <FIG>, or entirely separate processors, may be used to implement additional functionality in the vehicle <NUM> outside of the purposes of autonomous control, e.g., to control entertainment systems, to operate doors, lights, convenience features, etc..

In addition, for additional storage, the vehicle <NUM> may include one or more mass storage devices, e.g., a removable disk drive, a hard disk drive, a direct access storage device ("DASD"), an optical drive (e.g., a CD drive, a DVD drive, etc.), a solid state storage drive ("SSD"), network attached storage, a storage area network, and/or a tape drive, among others.

Furthermore, the vehicle <NUM> may include a user interface <NUM> to enable vehicle <NUM> to receive a number of inputs from and generate outputs for a user or operator, e.g., one or more displays, touchscreens, voice and/or gesture interfaces, buttons and other tactile controls, etc. Otherwise, user input may be received via another computer or electronic device, e.g., via an app on a mobile device or via a web interface.

Moreover, the vehicle <NUM> may include one or more network interfaces, e.g., network interface <NUM>, suitable for communicating with one or more networks <NUM> to permit the communication of information with other computers and electronic devices, including, for example, a central service, such as a cloud service, from which the vehicle <NUM> receives information including trained machine learning models and other data for use in autonomous control thereof. The one or more networks <NUM>, for example, may be a communication network and include a wide area network ("WAN") such as the Internet, one or more local area networks ("LANs") such as Wi-Fi LANs, mesh networks, etc., and one or more bus subsystems. The one or more networks <NUM> may optionally utilize one or more standard communication technologies, protocols, and/or inter-process communication techniques. In some implementations, data collected by the one or more sensors <NUM> can be uploaded to a computing system <NUM> via the network <NUM> for additional processing.

In the illustrated implementation, the vehicle <NUM> may communicate via the network <NUM> with a computing device <NUM> for the purposes of implementing various functions described below for generating simulation data and training machine learning models. In some implementations, computing device <NUM> is a cloud-based computing device. As described below in more detail with reference to <FIG>, the computing device <NUM> includes a simulation data generator <NUM> and a machine learning engine <NUM>. In some implementations not shown in <FIG>, the simulation data generator <NUM> may be configured and executed on a combination of the computing system <NUM> and the vehicle control system <NUM> of the vehicle <NUM>. For example, the simulation data generator <NUM> may execute some functionality on the vehicle control system <NUM> of the vehicle <NUM> while the simulation data generator <NUM> executes the remaining functionality on the computing system <NUM>. In other implementations, either the computing system <NUM> or the vehicle control system <NUM> of the vehicle <NUM> alone executes the functionality of the simulation data generator <NUM>. For example, in some implementations, the simulation data generator <NUM> operates on the computing system <NUM> to receive logged data from the memory <NUM> and generate simulation data that can be used to by the machine learning engine <NUM>. The machine learning engine <NUM>, operable on the computing system <NUM>, generates a machine learning model based on the simulation data. The machine learning model is sent from the computing system <NUM> to vehicle <NUM> to be used in the appropriate control subsystem <NUM>-<NUM> for use in performing its respective function.

Each processor illustrated in <FIG>, as well as various additional controllers and subsystems disclosed herein, generally operates under the control of an operating system and executes or otherwise relies upon various computer software applications, components, programs, objects, modules, data structures, etc., as will be described in greater detail below. Moreover, various applications, components, programs, objects, modules, etc. may also execute on one or more processors in another computer (e.g., computing system <NUM>) coupled to vehicle <NUM> via network <NUM>, e.g., in a distributed, cloud-based, or client-server computing environment, whereby the processing required to implement the functions of a computer program may be allocated to multiple computers and/or services over a network.

In general, the routines executed to implement the various implementations described herein, whether implemented as part of an operating system or a specific application, component, program, object, module or sequence of instructions, or even a subset thereof, will be referred to herein as "program code. " Program code typically comprises one or more instructions that are resident at various times in various memory and storage devices, and that, when read and executed by one or more processors, perform the steps necessary to execute steps or elements embodying the various aspects of the present disclosure. Moreover, while implementations have and hereinafter will be described in the context of fully functioning computers and systems, it will be appreciated that the various implementations described herein are capable of being distributed as a program product in a variety of forms, and that implementations can be implemented regardless of the particular type of computer readable media used to actually carry out the distribution.

Examples of computer readable media include tangible, non-transitory media such as volatile and non-volatile memory devices, floppy and other removable disks, solid state drives, hard disk drives, magnetic tape, and optical disks (e.g., CD-ROMs, DVDs, etc.) among others.

In addition, various program code described hereinafter may be identified based upon the application within which it is implemented in a specific implementation. However, it should be appreciated that any particular program nomenclature that follows is used merely for convenience, and thus the present disclosure should not be limited to use solely in any specific application identified and/or implied by such nomenclature. Furthermore, given the typically endless number of manners in which computer programs may be organized into routines, procedures, methods, modules, objects, and the like, as well as the various manners in which program functionality may be allocated among various software layers that are resident within a typical computer (e.g., operating systems, libraries, API's, applications, applets, etc.), it should be appreciated that the present disclosure is not limited to the specific organization and allocation of program functionality described herein.

The example environment illustrated in <FIG> is not intended to limit implementations disclosed herein. Indeed, other alternative hardware and/or software environments may be used without departing from the scope of implementations disclosed herein.

In some implementations, the computing system <NUM> receives a time-stamped log of vehicle data from the vehicle control system <NUM>. A time stamp can be added to each instance of vehicle data prior to uploading to computing system <NUM>. The logged data may include raw sensor data from any one or more of the sensors <NUM>, state or localization data from localization subsystem <NUM>, state or perception data from perception subsystem <NUM>, state or planning data from the planning subsystem <NUM> or state or control data from the control subsystem <NUM>. The logged data may optionally include other vehicle sensor data, logged sensor data, environmental data or identification data. In some instances, the logged data is collected using the autonomous vehicle <NUM> in its autonomous operations. In other instances, the logged data is collected using the autonomous vehicle <NUM> but when the autonomous vehicle <NUM> is being driven manually, and at least a subset of its sensors and modules are operating.

As examples, each instance of time-series log sensor data may include information on a location, orientation, and speed of the autonomous vehicle <NUM>. The tracking data for each instance of the time-series logged data may include tracking of objects external to the autonomous vehicle describing their position(s), extent(s), orientation(s), categories, speed(s), and other tracking data or tracking predictions. Information on static objects (e.g., highway signs, road surfaces, etc.) may also be logged. In some implementations, other forms of environmental data may also be logged (e.g., weather conditions, lighting conditions, visibility, etc.).

The logged data may be used as a source of data to aid in generating simulation scenarios. For example, in some implementations, an individual simulation scenario describes aspects of the motion behavior characteristics of the autonomous vehicle <NUM> (an ego-vehicle) and one or more actors (e.g., other vehicles, static environmental objects, and pedestrians) in an instantiation of a three-dimensional (3D) world within which the autonomous vehicle <NUM> interacts. In some implementations, an individual simulation may include a variety of simulation scenarios that describe a set of tests of different specific encounters between an autonomous vehicle, its environment, and other moving and non-moving actors (e.g., other vehicles, other autonomous vehicles, pedestrians, animals, machinery like traffic lights, gates, drawbridges, and non-human moveable things like debris, etc.).

<FIG> is a block diagram illustrating an example of the computing system <NUM> for generating simulation scenarios according to some implementations of this disclosure. More specifically, the simulation data generator <NUM> is used, for example, to generate a simulation scenario. In some implementations, the machine learning engine <NUM> may be used, for example, to train a machine learning model <NUM> using the simulation results of a simulation based on the simulation scenario.

Referring to <FIG>, the illustrated example computing system <NUM> includes one or more processors <NUM> in communication, via a communication system <NUM> (e.g., bus), with memory <NUM>, at least one network interface controller <NUM> with network interface port for connection to a network (e.g., network <NUM> via signal line <NUM>), a data storage <NUM>, other components, e.g., an input/output ("I/O") components interface <NUM> connecting to a display (not illustrated) and an input device (not illustrated), a machine learning engine <NUM>, and a simulation data generator <NUM>. Generally, the processor(s) <NUM> will execute instructions (or computer programs) received from memory <NUM>. The processor(s) <NUM> illustrated incorporate, or are directly connected to, cache memory <NUM>. In some instances, instructions are read from memory <NUM> into the cache memory <NUM> and executed by the processor(s) <NUM> from the cache memory <NUM>.

In more detail, the processor(s) <NUM> may be any logic circuitry that processes instructions, e.g., instructions fetched from the memory <NUM> or cache <NUM>. In some implementations, the processor(s) <NUM> are microprocessor units or special purpose processors. The computing device <NUM> may be based on any processor, or set of processors, capable of operating as described herein. The processor(s) <NUM> may be single core or multi-core processor(s). The processor(s) <NUM> may be multiple distinct processors.

The memory <NUM> may be any device suitable for storing computer readable data. The memory <NUM> may be a device with fixed storage or a device for reading removable storage media. Examples include all forms of non-volatile memory, media and memory devices, semiconductor memory devices (e.g., EPROM, EEPROM, SDRAM, and flash memory devices), magnetic disks, magneto optical disks, and optical discs (e.g., CD ROM, DVD-ROM, or Blu-Ray® discs). A computing system <NUM> may have any number of memory devices as the memory <NUM>. While the simulation data generator <NUM> and the machine learning engine <NUM> are illustrated as being separate from processor <NUM> and memory <NUM>, it will be appreciated that in some implementations, some or all of the functionality of the components <NUM> and <NUM> may be implemented with program code instructions resident in the memory <NUM> and executed by the processor <NUM>.

The cache memory <NUM> is generally a form of computer memory placed in close proximity to the processor(s) <NUM> for fast read times. In some implementations, the cache memory <NUM> is part of, or on the same chip as, the processor(s) <NUM>. In some implementations, there are multiple levels of cache <NUM>, e.g., L2 and L3 cache layers.

The network interface controller <NUM> manages data exchanges via the network interface (sometimes referred to as network interface ports). The network interface controller <NUM> handles the physical and data link layers of the OSI model for network communication. In some implementations, some of the network interface controller's tasks are handled by one or more of the processor(s) <NUM>. In some implementations, the network interface controller <NUM> is part of a processor <NUM>. In some implementations, a computing system <NUM> has multiple network interfaces controlled by a single controller <NUM>. In some implementations, a computing system <NUM> has multiple network interface controllers <NUM>. In some implementations, each network interface is a connection point for a physical network link (e.g., a cat-<NUM> Ethernet link). In some implementations, the network interface controller <NUM> supports wireless network connections and an interface port is a wireless (e.g., radio) receiver/transmitter (e.g., for any of the IEEE <NUM> protocols, near field communication "NFC", Bluetooth, ANT, WiMAX, <NUM>, or any other wireless protocol). In some implementations, the network interface controller <NUM> implements one or more network protocols such as Ethernet. Generally, a computing device <NUM> exchanges data with other computing devices via physical or wireless links (represented by signal line <NUM>) through a network interface. The network interface may link directly to another device or to another device via an intermediary device, e.g., a network device such as a hub, a bridge, a switch, or a router, connecting the computing device <NUM> to a data network such as the Internet.

The data storage <NUM> may be a non-transitory storage device that stores data for providing the functionality described herein. The data storage <NUM> may store, among other data, simulation data <NUM>, logged data <NUM>, augmented data <NUM>, and a machine learning model or representation <NUM>, as will be defined below.

The computing system <NUM> may include, or provide interfaces <NUM> for, one or more input or output ("I/O") devices. Input devices include, without limitation, keyboards, microphones, touch screens, foot pedals, sensors, MIDI devices, and pointing devices such as a mouse or trackball. Output devices include, without limitation, video displays, speakers, refreshable Braille terminal, lights, MIDI devices, and <NUM>-D or <NUM>-D printers. Other components may include an I/O interface, external serial device ports, and any additional coprocessors. For example, a computing system <NUM> may include an interface (e.g., a universal serial bus (USB) interface) for connecting input devices, output devices, or additional memory devices (e.g., portable flash drive or external media drive). In some implementations, a computing device <NUM> includes an additional device such as a co-processor, e.g., a math co-processor can assist the processor <NUM> with high precision or complex calculations.

In some implementations, the computing system <NUM> includes a machine learning engine <NUM> to train a machine learning model <NUM>. As shown in <FIG>, once the simulation data generator <NUM> has generated one or more simulation scenarios suitable for training the machine learning model <NUM>, the machine learning engine <NUM> may train the machine learning model <NUM> using the simulation scenarios as training examples. In one implementation, the machine learning model <NUM> is a neural network model and includes a layer and/or layers of memory units where memory units each have corresponding weights. A variety of neural network models can be utilized including feed forward neural networks, convolutional neural networks, recurrent neural networks, radial basis functions, other neural network models, as well as combinations of several neural networks. Additionally, or alternatively, the machine learning model <NUM> can represent a variety of machine learning techniques in addition to neural networks, for example, support vector machines, decision trees, Bayesian networks, random decision forests, k-nearest neighbors, linear regression, least squares, other machine learning techniques, and/or combinations of machine learning techniques.

One or more machine learning models <NUM> may be trained for a variety of autonomous vehicle tasks including determining a target autonomous vehicle location, generating one or more signals to control an autonomous vehicle, tracking or identifying objects within the environment of an autonomous vehicle, etc. These tasks can include various perception tasks or motion planning tasks. For example, a neural network model may be trained for perception tasks such as to identify traffic lights in the environment with the autonomous vehicle <NUM>. As a further example, a neural network model may be trained to predict the make and model of other vehicles in the environment with the autonomous vehicle <NUM>. Moreover, a neural network model may be trained for motion planning like to predict or forecast the future trajectory of other actors, predict future binary decisions made by other actors (e.g., will an actor change lanes, or will it yield), or trained to make control decisions for the autonomous vehicle (e.g., should the autonomous vehicle make a lane change or stop at a light or yield). In many implementations, machine learning models may be trained to perform a single task. In other implementations, machine learning models may be trained to perform multiple tasks.

The machine learning engine <NUM> generates training instances from the simulation scenarios to train the machine learning model <NUM>. A training instance can include, for example, an instance of simulated autonomous vehicle data where the autonomous vehicle <NUM> can detect a stop sign using the simulated sensor data from one or more sensors and a label corresponding to a simulated output corresponding to bringing the autonomous vehicle to a stop in the simulation scenario. The machine learning engine <NUM> applies a training instance as input to machine learning model <NUM>. In some implementations, the machine learning model <NUM> may be trained using any one of at least one of supervised learning (e.g., support vector machines, neural networks, logistic regression, linear regression, stacking, gradient boosting, etc.), unsupervised learning (e.g., clustering, neural networks, singular value decomposition, principle component analysis, etc.), or semi-supervised learning (e.g., generative models, transductive support vector machines, etc.). Additionally, or alternatively, machine learning models in accordance with some implementations may be deep learning networks including recurrent neural networks, convolutional neural networks (CNN), networks that are a combination of multiple networks, etc. For example, the machine learning engine <NUM> may generate a predicted machine learning model output by applying training input to the machine learning model <NUM>. Additionally, or alternatively, the machine learning engine <NUM> may compare the predicted machine learning model output with a machine learning model known output (e.g., simulated output in the simulation scenario) from the training instance and, using the comparison, update one or more weights in the machine learning model <NUM>. In some implementations, one or more weights may be updated by backpropagating the difference over the entire machine learning model <NUM>.

The machine learning engine <NUM> may test a trained machine learning model according to some implementations. The machine learning engine <NUM> may generate testing instances using the simulation scenarios and the simulated autonomous vehicle in the simulation scenario performing the specific autonomous vehicle task for which the machine learning model <NUM> is trained. The machine learning engine <NUM> may apply a testing instance as input to the trained machine learning model <NUM>. A predicted output generated by applying a testing instance to the trained machine learning model <NUM> may be compared with a known output for the testing instance (i.e., a simulated output observed in the simulation) to update an accuracy value (e.g., an accuracy percentage) for the machine learning model <NUM>.

In some implementations, the simulation data generator <NUM> converts the logged data accessible in the logged data <NUM> of the data storage <NUM> in different ways to generate simulation data <NUM>. For example, the logged data is used as a source of data that is based on ground level truth about real world driving situations to generate simulation data stored in simulation data <NUM> of the data storage <NUM>. In many implementations, the simulation data <NUM> represents an editable source of truth defining a number of simulation scenarios. The simulation data may, for example, be used in simulations of a perception subsystem or a planning model. However, more generally, the simulation data <NUM> could be used for other purposes, such as procedural scene generation as one example. In some implementations, one or more components of an instance of the logged data <NUM> are used to aid in creating at least one aspect of a simulation scenario. For example, in some implementations, the logged data <NUM> is used as an aid to generate a description including a behavior, vehicle configuration (e.g., autonomous vehicle location, platform, speed, or orientation), and sensor configuration of autonomous vehicle (e.g., ego vehicle) and the environment including actors (e.g., other vehicles, traffic, pedestrians, and static objects) in a simulation scenario. However, more generally, in some implementations, other information available from the logged data <NUM> may be used as an aid in generating a simulation scenario. The logged data <NUM> may be generally used, in some implementations, as a resource to provide a source of real sensor data for a simulation task that requires a source of real sensor data.

In some implementations, the simulation data <NUM> is used run simulations that, in turn, are used to generate training data for the machine learning engine <NUM>. In some implementations, the trained machine learning model <NUM> may be used in the autonomous vehicle <NUM> for performing various autonomous vehicle tasks relating to perception, planning, and control, among other things.

An appropriate dataset of quality training data is needed to learn autonomous vehicle tasks. For example, autonomous vehicle tasks may include control signals indicating a route change action, a planning action, and/or other autonomous vehicle actions which are generated in response to data collected from one or more autonomous vehicle sensors. Waiting for real world sensor data to be gathered for use as training data for autonomous vehicle tasks may take extended periods of time (e.g., months, years, etc.). Additionally, other sources of training data, such as video game engines or video/film data, typically don't provide training data that is realistic.

Generating simulation scenarios based on logged data <NUM> has an advantage in that the simulation scenarios may be highly realistic because they are based off of logged data <NUM>. Additionally, as described below in more detail, many variations on the simulation scenarios may be generated to increase the variety and quantity of training data. The simulation scenarios generated from logged data <NUM> may generally be used to simulate an encounter between the autonomous vehicle <NUM>, its surrounding environment, and other entities (i.e., other actors) in the surrounding environment. In some implementations, the logged data <NUM> may be used to generate variations in simulation scenarios. In some implementations, the variations in the simulation scenarios are generated algorithmically. For example, the process starts with a scenario, changes the types of actors, adding/subtracting them, and using different parameters for models that control the actors. The variations of the simulation scenarios can also be generated algorithmically by varying geometry of the world, traffic lights, paint, geometric models of the models, their light, any other properties of the autonomous vehicle's environment. The simulation scenarios may provide a dataset that includes information to instantiate a three-dimensional world that mimics the motion behavior and sensor configuration of the autonomous vehicle <NUM>, other vehicles (autonomous and/or non-autonomous), and pedestrians, among other things.

In implementations consistent with the disclosure, the simulation data generator <NUM> may include: a data mapping engine <NUM>, an augmentation engine <NUM> that generates augmented data <NUM>, a scenario production engine <NUM>, and optionally a simulator <NUM>. The data mapping engine <NUM>, the augmentation engine <NUM>, the scenario production engine <NUM> and the simulator <NUM> in the simulation data generator <NUM> are example components in which techniques described herein may be implemented and/or with which other systems, components, and techniques described herein may interface. The operations performed by one or more engines <NUM>, <NUM>, <NUM>, and the simulator <NUM> of <FIG> may be distributed across multiple computing systems. In some implementations, one or more aspects of engines <NUM>, <NUM>, <NUM>, and may be combined into a single system and/or one or more aspects may be implemented by the computing system <NUM>. Engines in accordance with many implementations may each be implemented in one or more computing devices that communicate, for example, through the communication network <NUM>. It should be noted that the simulator <NUM> is illustrated with dashed lines to indicate that is an optional component of the simulation data generator <NUM>, and in other implementations it may coupled for communication with the simulation data generator <NUM> via the communication system <NUM> or may be part of another computing system (not shown) entirely and coupled for communication via signal line <NUM> and the network controller <NUM>.

The data mapping engine <NUM> may access and process the logged data <NUM> and perform one or more operations to map the logged data <NUM> into an initial form that identifies actors, actor types, and actor motion behavior characteristics (e.g., actor trajectories, including actor speed). For example, in some implementations, the logged data <NUM> includes perception data from a perception subsystem <NUM> that includes tracks or tracking data that are predictions on directions, shapes, speeds, sizes, and types of tracked objects. The logged data <NUM> may also include an output of a localization subsystem <NUM>, describing location information for the ego-vehicle.

In some implementations, the data mapping engine <NUM> maps a time-series sequence of instances of the logged data <NUM> to a global coordinate system. Optional smoothing of the mapped time-series data may be performed in some implementations to reduce noise. The identified actors may be fit to a movement model to estimate their movement. In some implementations, smoothing is performed from simulations data.

In some implementations, the tracking data includes a track ID for each tracked object e.g., a unique ID for a tracked object). The tracking data may, for example, include a track ID, size, type, and bounding box. In some implementations, the data mapping engine <NUM> identifies actors by determining whether or not a track with an ID that occurs over a sequence of instances of logged data is a unique actor. For example, criteria for determining that a sequence of instances of a track ID is a unique actor may include rules on a minimum number of instances that the track with a particular ID occurs, rules based on a consistency with the track ID repeats in a sequence, etc..

In some implementations, the identified actors are fit to a movement model to estimate their movement. The mapping may include one or more rules to generate an output that identifies a set of actors (e.g., vehicles, pedestrians, and static objects) and actor states. The actor states include actor motion behavior characteristics, for example, an orientation; speed, location in the global coordinate system, pose, and derivatives of the actor (e.g., acceleration). The actor motion behavior characteristics correspond to a trajectory traversed by the actor in an environment about an autonomous vehicle (the ego-vehicle).

Each actor has an associated actor type (e.g., an actor type corresponds to an object type, such as pedestrians; different types of vehicles such as cars, trucks, motorcycles, bicycles; and may also optionally in some implementations include static environmental objects). Each actor has various properties that may be varied including geometric shape, color, reflectivity, size, orientations, and any other specific perception properties of actors. The actor type may also be considered to be an actor state, in that in some implementations the actor type may be changed, as discussed below in more detail. The output of the mapping may also, in some implementations, identify an ego-vehicle state describing the motion of the ego-vehicle, which in some implementation may include the location, pose, and speed of the ego-vehicle in the global coordinate system.

The augmentation engine <NUM> samples the actor states and the ego-vehicle state and generates augmented data. In some implementations, the augmentation engine <NUM> manipulates the identified actors and actor states (e.g., actor types, and actor motion behavior characteristics, such as the trajectory) to generate variations. The process of manipulating or modifying the actor information may also be called mutation. In some implementations, the output of the augmented engine <NUM> includes a set of actors, actor types, and associated actor motion behavior characteristics that may have one or more attributes varied in comparison with the original set of actors, actor types, and associated actor motion behavior characteristics. In some implementations, the augmentation engine <NUM> may be implemented to generate specific mutations in response to configurable input criteria. Other possibilities include generating a wide range of mutations and outputting specific mutations that correspond to configurable input criteria. Some examples of manipulations that may be performed by the augmentation engine <NUM> include changing a speed or acceleration of an actor, changing the actor type or size, changing an offset position (e.g., a lateral or longitudinal offset) of an actor, changing the trajectory of an actor, digitally adding or deleting actors, changing the motion behavior characteristics of an actor, or any permutations, combinations, and variations thereof. Specific example manipulations are described in more detail below with reference to <FIG>. In some implementations, the augmentation engine <NUM> may also modify environment, goals and assumptions. The process of manipulation may also be configured over a configurable range of possible supported variations.

In some implementations, the scenario production engine <NUM> generates simulation scenarios based on the augmented data <NUM>. In some implementations, a scenario includes information describing one or more actors; an actor type for each actor; and actor motion behavior characteristics for each actor. For example, a typical simulation scenario specifies dynamic motion behavior characteristics including behavior characteristics relevant to how an actor interacts with a simulated autonomous vehicle and other actors in a simulated 3D world. The simulation scenarios may also include the initial conditions, a timeline of significant events and the related environmental conditions, but also the simulator configuration. In some implementations, the scenario production engine <NUM> generates a platform file <NUM> (See <FIG> below) describing at least one configuration of an autonomous vehicle and actors. For example, a scenario may include an ego-vehicle state describing its speed, an ego-vehicle local pose, actor dynamic states, calibration data (for executing simulations), and configuration data (for executing simulations). For example, a perception system simulation may require calibration data and configuration data for some aspects of the simulation of a particular perception system, such as its LIDAR system. In some implementations, a selectable range of variations in a configuration is supported by the platform file <NUM>. In addition to configuration information, additional code or instructions may be included in the platform file <NUM> for use in generating simulations. In some implementations, the platform file <NUM> includes a configuration file that defines input files, configured variations of targets in the augmented data, metadata tags to define attributes of added actors such as a number of pedestrians, and other information required to generate changes in state in the scenario. In some implementations, the scenario production engine <NUM> may register a simulation scenario by generating a simulation identifier, assigning the simulation identifier to the simulation scenario, and storing the simulation scenario in the simulation data <NUM>. For example, the simulation identifier may be a globally unique identifier (GUID). The simulation data <NUM> may be a database storing currently and previously available simulation scenarios indexed by their corresponding simulation identifiers.

A final component of the simulation data generator <NUM> is the simulator <NUM>. The simulator <NUM> may execute a simulation based on a selected simulation scenario. For example, the simulation scenario may correspond to a perception simulation scenario that imitates the operation of the perception subsystem <NUM> or a planning simulation scenario that imitates the operation of the planning subsystem <NUM> of the autonomous vehicle <NUM>. In some implementations, the scenario production engine <NUM> sends a simulation identifier to the simulator <NUM>. The simulator <NUM> uses the simulation identifier to fetch a configuration of a matching simulation scenario from the simulation data <NUM> and executes a simulation based on the fetched simulation scenario configuration. The simulator <NUM> may create a run identifier (run ID) to associate with an execution (run) of the simulation. In some implementations, the simulator <NUM> may create a batch of a plurality of simulation scenario variations and execute the batch in a single execution. In such implementations, the simulator <NUM> may create a batch identifier (batch ID) to associate with the batch execution. The simulator <NUM> may generate a simulation result and/or a simulation log during the execution of the simulation and store it in the simulation data <NUM>. In some implementations, the simulation result and/or a simulation log are one or more formatted messages including or encoded with state information of the autonomous vehicle <NUM> and other actors observed in the simulation. The simulation log may be stored in the database of simulation data <NUM> storing a historical log of simulation runs indexed by corresponding run ID and/or batch ID. More generally, the simulation result and/or a simulation log may be used as training data for machine learning engine <NUM>.

As illustrated in <FIG>, in some implementations the data mapping engine <NUM> may select from different snippets 302a, 302b. 302n of logged data <NUM>. Additionally, <FIG> illustrates a particular advantage of the present disclosure, that it may generate the simulation scenario from real logged sensor data (e.g., snippets 302a, 302b. 302n of logged data) combined with other simulated data from non-sensor data sources <NUM> and other atlas or road map data. For example, there may also be some association of actors with roadway information. Similarly, the logged data may be associated with roadway information. In particular, the non-sensor data sources <NUM> may include data from video games or data from film or video. This non-sensor data, while less realistic, may also be mapped to the same set of global coordinates and be used as an additional source of data. More specifically, the data input to the data mapping engine <NUM> may be any combination of real logged data, video game data and film data. For example, an individual scenario may be based on sampling a snippet <NUM> of a much larger set of logged data <NUM>. Snippets 302a, 302b. 302n of logged data <NUM> may be selected for use in generating a simulation scenario in different ways. For example, snippets <NUM> of logged data may include an identifier or tag identifying portions of the logged data of potential interest for generating simulation scenarios. For example, ID tags may be added, while collecting logged data, to identify one or more of a geography (e.g., San Francisco, New York, etc.), actors (e.g., other vehicles, bicycles, pedestrians, mobility scooters, motorized scooters, etc.), behaviors (e.g., lane change, merge, steering, etc.), location (e.g., four-way stop, intersection, ramp, etc.), status (e.g., deprecated, quarantined, etc.), etc. Alternatively, snippets <NUM> of logged data may be selected in other ways, such as by using a search tool to search for specific characteristics of portions of the logged data. Other approaches are also possible to select a snippet of logged data, including random selection techniques. Similarly, portions or snippets of the non-sensor data sources <NUM> may labeled with the same identifiers or tags based on geography, actors, behaviors, location, status, state, etc. <FIG> also illustrates the data flow through the simulation data generator <NUM> for the components of the data mapping engine <NUM>, the augmentation engine <NUM>, the scenario production engine <NUM>, and optionally the simulator <NUM>. As shown, once the different snippets 302a, 302b. 302n of logged data <NUM> and snippets from the non-sensor data sources <NUM> are selected and processed by the data mapping engine <NUM>, the data mapping engine <NUM> sends the data for the identified actors, actor types, and actor motion behavior characteristics to the augmentation engine <NUM>. As described above, the augmentation engine <NUM> receives that input and generates augmented data <NUM>. The augmentation engine <NUM> outputs the augmented data <NUM> to the scenario production engineer <NUM> which generates one or more simulations scenarios. The scenario production engineer <NUM> provides the one or more simulations scenarios to the simulator <NUM> that executes the simulations defined by the one or more simulations scenarios and the execution of the simulator <NUM> produces simulations results or messages. These results or message can be stored in the data store for further analysis and/or use as training data.

Referring now to <FIG>, an example of the augmentation engine <NUM> according to some implementations is illustrated. The augmentation engine <NUM> may include a tracking data processing engine <NUM>, an ego-vehicle state engine <NUM>, an actor state engine <NUM>, an actor attribute manipulation engine <NUM>, an ego-vehicle attribute manipulation engine <NUM>, and a scene data manipulation engine <NUM>.

In some implementations, the tracking data processing engine <NUM> performs an initial identification of actors and actor states from the mapped tracking data. An ego-vehicle state engine <NUM> is included in some implementations to determine a state of the ego-vehicle, such as an ego-vehicle location, pose, speed, etc. An actor state engine <NUM> is included in some implementations to manage actor states, such as actors, actor speeds, actor types, etc. An actor attribute manipulation engine <NUM> is included in some implementations to manipulate actor states and generate variations. Additional optional engines may be provided to perform other types of state manipulation. For example, to the extent that manipulation in ego-vehicle states is desired, such as ego-vehicle speed, an ego-vehicle attribute manipulation engine <NUM> may be provided to manipulate ego-vehicle states. In some implementations, other aspects of a scene may be manipulated, such as adding sidewalks for pedestrians. A scene data manipulation engine <NUM> may be provided to implement manipulations of the environment in a scene. Examples of specific manipulations that are performed respectively by the tracking data processing engine <NUM>, the ego-vehicle state engine <NUM>, the actor state engine <NUM>, the actor attribute manipulation engine <NUM>, the ego-vehicle attribute manipulation engine <NUM>, and the scene data manipulation engine <NUM> will be described in detail below with reference to <FIG>. From the description below, the processes, changes and modifications that each of these engines <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> make to the data input from the data mapping engine <NUM> can be easily understood.

Referring now to <FIG>, an example of the scenario production engine <NUM> according to some implementations is illustrated. The scenario production engine <NUM> receives the augmented data <NUM> from the augmentation engine <NUM> or retrieves it from data storage <NUM>. The scenario production engine <NUM> process the augmented data <NUM> to generate one or more simulation scenarios. In some implementations, the simulation scenarios generated by the scenario production engine <NUM> take the form of a platform file <NUM>. A platform file <NUM> of a simulation scenario may be implemented in different ways. The platform file <NUM> may be a single file as shown in <FIG> or a group of files each storing a different type of data as described below. In the example shown in <FIG>, the platform file <NUM> includes vehicle data <NUM> describing information on vehicles and other actors in scenario (e.g., actors), calibration data <NUM> for variables that require calibration required to execute the simulation, assets <NUM> for machine learning (e.g., resources for machine learning), simulation configuration data <NUM> that specifies the configuration information for a simulation, and optionally a file management data file <NUM> for general management functions. In some implementations, the calibration data <NUM> calibrates an attribute of the ego-vehicle or an actor. The configuration data <NUM> is used to configure different aspects of simulations. That is, the platform file <NUM> may include information and code that subsequent simulations use as an aid to generate and execute one or more simulations.

Referring now to <FIG>, a block diagram illustrating an example of a data flow through the simulation data generator <NUM>, the simulator <NUM>, and the machine learning engine <NUM> will be described. The logged data is received by the simulation data generator <NUM>, which generates simulation data (e.g., one or more simulation scenarios). The simulator <NUM> receives simulation data or simulation scenario as has been described above and executes a simulation based on the simulation scenario. This may include simulations to evaluate components of the autonomous vehicle <NUM>, such as a simulation of a perception subsystem <NUM> or a planning subsystem <NUM>. The execution of the simulation generates simulation results or messages encoded with state information associated with the behavior of the autonomous vehicle <NUM> and other actors in the simulation scenario. The results/messages from the simulator <NUM> are used as a source of training data for a machine learning engine <NUM> used to train machine learning model <NUM>. The machine learning engine <NUM> retrieves a base model <NUM> and uses the simulation data to train the base model <NUM> and generate a trained machine learning model <NUM>. The simulation data may be repeatedly and iteratively used to improve the accuracy of the machine learning model <NUM> as represented by line <NUM> to and from the machine learning engine <NUM> in <FIG>. Thus, the simulation data may also be used for re-training or refinement of the machine learning model <NUM>. The improved machine learning model <NUM> can in turn be used by the perception subsystem <NUM>, for example. Various other specific parameters of any of the machine learning models <NUM> for perception, location, planning or control may be similarly trained or refined using validated data generated specifically for a particular parameter by the computing system <NUM>.

In some implementations, the machine learning engine <NUM> may generate training instances to train a neural network model. For example, in some implementations, the simulation result may be used by machine learning engine <NUM> to generate updates with respect to a base model <NUM>. In some implementations, the simulation results are used to generate a predicted output of the machine learning model <NUM>, which is then used to update one or more weights in the machine learning model <NUM> by determining a difference between the predicted output and a simulated output.

In some implementations, the machine learning model <NUM> is a neural network model. Additionally, or alternatively, the neural network engine <NUM> may compare the predicted neural network model output with a neural network model known output (e.g., simulated output generated from the simulation scenario) from the training instance and, using the comparison, update one or more weights in the neural network model. In some implementations, one or more weights may be updated by backpropagating the difference over the entire neural network model.

In a variety of implementations, a neural network model can be trained using supervised learning, unsupervised learning, and semi-supervised learning. Additionally, or alternatively, neural network models in accordance with some implementations can be deep learning networks including recurrent neural networks, convolutional neural networks, networks that are a combination of multiple networks, etc..

In some implementations, one or more aspects of the simulation scenario may be selected to aid in generating a wider variety of instances of training data. For example, one or more scenarios may have a configurable range of variations in the speed of actors selected to increase the variety of training data used to train the machine learning model <NUM>. As a simple example, varying the speed of actors in a scenario may be useful to train the machine learning model <NUM> to make one or more predictions over a range of actor speeds that may not have occurred in the original logged data.

As another example, changing the actor type in a scenario may be useful to train the machine learning model <NUM> to make predictions over a range of actor types that may not have occurred in the original logged data.

As an illustrative but non-limiting example, increasing the speed of actors may be useful for a variety of purposes, such as generating simulations indicative of how well a simulated perception subsystem <NUM> makes predictions for different scenarios, such as making predictions about the detection of simulated actors or the detection of attributes of simulated actors (e.g., a detection of the brake lights of a simulated actor). For example, increasing a speed of an actor or changing it to a different actor type (e.g., changing it from a truck to a motorcycle) may pose a greater detection challenge to a perception subsystem <NUM>.

As another illustrative but non limiting example, changing the behavioral response of an actor, such as changing its perception range or intelligence, may be useful for generating simulations to test how well a planning subsystem <NUM> makes predictions for responding to different situations.

In some implementations, the selection of the logged data and the manipulations performed by the augmentation engine <NUM> may be selected to further one or more aspects of training the machine learning model <NUM>.

<FIG> are diagrams that illustrate graphical representations of the simulated data or mutated simulated data. The example of simulated data in <FIG> and <FIG> are generated from logged data in accordance with the implementations that have been described above. For the purposes of illustration, an example of an autonomous vehicle <NUM> making a right turn at an intersection is illustrated, although it will be understood that variations may be generated from other different possible starting situations including but not limited to examples of an ego-vehicle making a left turn, an ego-vehicle continuing straight on a road, a ego-vehicle stopping or restarting, etc. The simulated data also includes another actor vehicle that was detected by the sensors of the autonomous vehicle <NUM> as represented in the logged data. <FIG><FIG> and <FIG> are diagrams that illustrate mutations of the simulated data (generated from logged data of <FIG> or <FIG>) where the mutations vary different aspects of actors and behaviors of actors.

<FIG> illustrates an example representation <NUM> of the simulated data that has been generated from the logged data. The representation <NUM> shows arrows for the motion of the autonomous vehicle <NUM> based on the logged data. In this example, the motion of an actor vehicle XX <NUM> is illustrated by the lines 706a, 706b from point A to point B and point B to point C, which is based on the logged data. Actor vehicle XX <NUM> has an associated shape and size associated with its actor type, as illustrated by the rectangular box. In this example, actor vehicle XX <NUM> traverses a path from position A to position B shown by line 706a and then from position B to position C shown by line 706b. The motion of the autonomous vehicle <NUM> is represented by lines <NUM> showing in a right turn.

Some examples of manipulations that may be performed by the augmentation engine <NUM> include changing a speed or acceleration of an actor. For example, starting with the actor velocity or acceleration, the actor's motion along its tracked trajectory may be manipulated to have a faster speed, greater acceleration, slower speed, or slower acceleration. <FIG> illustrates an example representation <NUM> of mutated simulation data produced by the augmentation engine <NUM> from the simulation data of <FIG>. In the representation <NUM> of mutated simulation data, the actor vehicle XX <NUM> speed is manipulated to be faster between positions B and C as represented by dashed line <NUM>. Similarly, in another example of mutated simulation data, the actor vehicle XX <NUM> speed is manipulated to be slower between positions B and C. <FIG> shows the representation <NUM> of mutated simulation data where the actor vehicle XX <NUM> speed is manipulated to be slower between positions B and C as represented by dashed line <NUM>.

Other examples of manipulation include changing an actor type from one actor type <NUM> to another actor type <NUM>, <NUM>. For example, suppose an actor type <NUM> had a size and shape corresponding to that of a car. The actor type <NUM> may be changed to that of an actor having a different size and shape, such as that of a motorcycle or a truck. Other possibilities include changing a vehicle to a pedestrian. Again, <FIG> and <FIG> illustrate examples representations <NUM>, <NUM> of mutated simulation data produced by the augmentation engine <NUM> from the simulation data of <FIG>. <FIG> shows a representation <NUM> of mutated simulation data produced from the simulation data of <FIG> in which the actor vehicle type <NUM> is mutated to be a different type of vehicle, YY, <NUM>. For example, a car vehicle type <NUM> may be mutated to be a motorcycle vehicle type <NUM>. <FIG> shows a representation <NUM> of mutated simulation data produced from the simulation data of <FIG> in which the actor vehicle type <NUM> is mutated to be a different size of object, vehicle, ZZ, <NUM>. This may include, for example, variations in length, width or height of an actor that is a vehicle. For example, a vehicle type that is a car may be mutated to be a different size of truck, such as a longer or shorter truck. In this example, the mutation results in a larger size of the object, vehicle ZZ <NUM>, although more generally, the mutation may result in a smaller size of the object. From the example mutations of <FIG> and <FIG>, it can be understood how any type or characteristic of an actor type may be changed from the simulation data by the augmentation engine <NUM>.

Other examples of manipulations that may be performed by the augmentation engine <NUM> include manipulating the path of an actor. For example, a selectable offset <NUM>, <NUM> in position (e.g., a lateral or longitudinal offset) may be introduced in to the trajectory of an actor. Variations may be introduced into the trajectory of an actor, such as by adding a time varying signal or noise source into the trajectory of the actor, such as adding a variance in position or speed. For example, suppose an actor was moving in a comparatively straight trajectory with a nearly constant speed. A variance in position or speed within a selected maximum swing may be added to the position or speed of the actor about the original trajectory. For example, an additional controller signal may simulate physically realistic variations of position and speed with a maximum range of variations (e.g., a ± <NUM>% variation, a ± <NUM>% variation, a ± <NUM>% variation, etc.). Again, <FIG> and <FIG> illustrate examples representations <NUM>, <NUM> of mutated simulation data produced by the augmentation engine <NUM> from the simulation data of <FIG>. <FIG> shows a representation <NUM> of mutated simulation data produced from the simulation data of <FIG> in which the actor vehicle XX <NUM> is mutated to have a lateral offset <NUM> of the path 706a, 706b to a new path <NUM>. For example, the mutation may introduce a selectable offset <NUM> to shift the actor vehicle XX <NUM> from one traffic lane or to an adjacent traffic lane. <FIG> shows a representation <NUM> of mutated simulated data produced from the simulation data of <FIG> in which the simulated data is mutated to also have a longitudinal offset <NUM> of path <NUM> in addition to the lateral offset of <FIG>. <FIG> illustrates not only that a longitudinal offset <NUM> can be added, but also that multiple offsets can be combined.

As another example, the behavior of an actor may simulated to follow its original trajectory (based on the logged data) up to a certain point (e.g., point B) and then the rules governing the actor's motion may be varied (e.g., making the actor to be no longer bound by the original trajectory). For example, the path of the actor may be modified by introducing direction changes or variations or modifying the trajectory to follow breadcrumbs or different points from the starting point to the end point. <FIG> and <FIG> illustrate examples representations <NUM>, <NUM> of mutated simulation data produced by the augmentation engine <NUM> from the simulation data of <FIG> for such variations. For example, a motion planning function may be turned on or off for the actor. <FIG> shows the representation <NUM> of mutated simulation data produced from the simulation data of <FIG> in which the actor vehicle XX <NUM> is mutated to have minor variations in position and speed on the path <NUM> between points B and C. Likewise, <FIG> shows the representation <NUM> of mutated simulation data produced from the simulation data of <FIG> in which the actor vehicle XX <NUM> is mutated to have major variations in position and speed on the path <NUM> between points B and C.

Other examples of mutation or manipulation also include digitally adding or deleting actors. For example, suppose that there were two cars and one truck identified from the original log sensor data. One of the two cars may be digitally deleted. Another vehicle, such as a motorcycle, may be digitally added. As another example, pedestrians may also be added. <FIG> illustrates an example representation <NUM> of the simulated data that has been generated from the logged data. <FIG> is similar to <FIG>; however, it includes an additional actor LL <NUM> on a path <NUM> based on the logged data. The representation <NUM> again shows arrows for the motion of the autonomous vehicle <NUM> based on the logged data. The motion actor vehicle XX <NUM> traverses a path illustrated by the lines 706a, 706b from point A to point B and point B to point C, respectively, which is based on the logged data. Actor vehicle XX <NUM> has an associated shape and size associated with its actor type, as illustrated by the box. The motion of the autonomous vehicle <NUM> is represented by lines <NUM> showing in a right turn. Actor LL <NUM> has an associated shape and size associated with its actor type and is illustrated by the hexagon. Actor LL <NUM> traverses a straight path as shown by line <NUM>. <FIG> shows the representation <NUM> of mutated simulation data produced from the simulation data of <FIG> which has been mutated in a number of different ways. First, an actor, LL <NUM>, has been deleted, an actor MM <NUM> has been added, and motion of actor XX has been modified between points B and C. This again illustrates how any number of modifications or mutations of the original simulation data generated from logged data may be changed in different aspects to create even more simulation scenarios. <FIG> is also provided to illustrate how actors can be deleted or added to produce mutated simulation data. As shown in <FIG>, in this example, actor LL <NUM> is deleted <NUM>. Also, a new actor MM <NUM> has been added. Actor MM <NUM> has an associated shape and size associated with its actor type and is illustrated by the diamond on a path <NUM> with a position below and following the motion of the autonomous vehicle <NUM>. Other mutations are also possible, such as mutating the motion of actor vehicle XX <NUM>, such as varying <NUM> its motion after point B to point C so that it is not be based on the tracking data.

Other examples include changing the motion behavior characteristics of an actor in the sense of changing the rules regarding how an actor perceives and interacts with other actors and with the autonomous vehicle <NUM>. For example, a perception range of an actor may be varied. As another example of manipulations, an actor may have different rules governing how intelligently it interacts with other actors or with the ego-vehicle. For example, different rule sets may be applied regarding how actor-vehicles interact with pedestrians. In some implementations, an actor may be a path follower. In some implementations, an actor may be a breadcrumb follower in the sense that it is attracted to an actor's breadcrumb in a simulation scenario. <FIG> shows the representation <NUM> of simulated data that has been mutated by changing an actor's behavioral response characteristics. In this example, the actor vehicle XX <NUM> motion from points A to B is based on tracking data. After point B, the actor vehicle XX <NUM> motion may be based in part on being based on the actor being assigned an actor type having a motion behavioral response. For example, an actor XX <NUM> might vary its speed and position after point B based on the presence of other actors that are nearby. Or the actor's motion response might be varied in other ways, such that its motion is no longer based on the logged data after point B. The change in actor behavioral response attribute (e.g., intelligence/perception) of the actor vehicle XX <NUM> from point B to point C is shown by the crosshatched area <NUM> of <FIG>.

Additionally, many permutations, combinations, and variations are possible. For example, the augmentation engine <NUM> may implement combinations such as varying a first actor's speed, deleting a second actor, changing a third actor to a different actor type, etc. As another example, an actor's speed may be varied and the actor type may both be changed. Thus, for example, if the originally identified actor was a car moving at some initial speed, the manipulation may change the actor type to a truck moving at twice the initial speed.

The process of manipulation may also be configured over a configurable range of all possible supported variations. For example, a speed of an actor may be manipulated over a range of speeds. The actor type may be manipulated over a range of actor types. The actor size may be manipulated over a range of actor sizes. Lateral or longitudinal offsets may be manipulated over a range of lateral and longitudinal offsets. A noise variance in speed and position along a path may be varied over a range. An actor's perception range may also be varied over a range of perception ranges. Other aspects of the behavioral response of an actor may be varied over a supported range of possibilities, such as a supported range of actor intelligence types. These modifications are all within the scope of the present disclosure.

<FIG> illustrates a flowchart of a method <NUM> in accordance with some implementations. In block <NUM>, logged data <NUM> is received or retrieved. As noted above, for example, the logged data <NUM> may include raw sensor data from any one or more of the sensors <NUM>, state or localization data from localization subsystem <NUM>, state or perception data from perception subsystem <NUM>, state or planning data from the planning subsystem <NUM> or state or control data from the control subsystem <NUM>. In some examples, the logged data is received as it is collected. In other examples, the logged data <NUM> is retrieved from the data storage <NUM>. In block <NUM>, the logged data <NUM> is mapped into a coordinate system of a simulation. In some implementations, the logged data <NUM> is mapped into global coordinates in order to map the motion of the ego-vehicle and actors in the global coordinate system. An example coordinate system is an orthogonal curvilinear coordinate (OCC) system. In block <NUM>, optional smoothing is performed of the mapped logged data to generate smoothed data. This may include noise-filtering or interpolation as examples. In some implementations, smoothing is performed from simulations data at a later stage. The smoothed data or the mapped logged data is then used to create the augmented data. In block <NUM>, in some implementations, the method <NUM> identifies actors and generates actor states to create initial augmented data. For example, initial actor information is identified from the mapped and smoothed logged data. This may include one or more rules regarding identifying actors based on verifying that an actor occurs in a meaningful number and frequency of instances of the logged data. In some implementations, the identified actors are fit to a movement model to estimate their movement. This creates an initial form of the augmented data. In block <NUM>, the initial augmented data of block <NUM> is sampled. For example, not all of the initial actor information may be required to generate a variation of the augmented data. For example, it may be desired to vary aspects of individual actors. So individual actors can be sampled, certain behaviors can be sampled, or locations, poses, other variable in the simulation can be sampled from the full set of logged data <NUM> that has been converted to the initial augmented data. In block <NUM>, one or more variations of the sampled augmented data are generated. For example, configuration variables may be used to define the one or more variations. Examples of such variations have been described above with reference to <FIG>. It should be noted that block <NUM> includes generating a plurality of sets of sampled augmented data where each set of sampled data corresponds to a variation in one or more actors and one or more characteristics or behavior. In block <NUM>, one or more simulation scenarios are generated based on the augmented data and its variations. For example, one simulation scenario may be generated for each set of sampled augmented data. Again, in some implementations, the variations in the simulation scenarios are generated algorithmically. For example, the process starts with a scenario, changes the types of actors, adding/subtracting them, and using different parameters for models that control the actors. The variations of the simulation scenarios can also be generated algorithmically by varying geometry of the world, traffic lights, paint, geometric models of the models, their light, any other properties of the autonomous vehicle's environment. In block <NUM>, one or more simulations are run using the one or more simulation scenarios.

In some implementations, the simulation data generator <NUM> is part of a larger pipeline <NUM>. <FIG> illustrates an example of a pipeline <NUM> of a simulator data generator <NUM> in accordance with some implementations. In some implementations, individual modules of the pipeline <NUM> may be implemented in software or firmware. In one implementation, an augmented data generation block <NUM> performs the data mapping and generation of augmented data. It receives the time-series logged data and generates the augmented data. The augmented data is represented in a global coordinate system using variables interpretable by other entities in the computing system <NUM>.

A simulation scenario generation block <NUM> generates simulation scenarios. In some implementations, the scenarios are generated at least in part based on the augmented data. In some implementations, an individual simulation may include a variety of simulation scenarios that describe a set of tests of different specific encounters between an autonomous vehicle <NUM>, its environment, and other actors.

In some implementations, a perception system simulation block <NUM> generates configuration information, state information and other information need to simulate the state and operation of the perception system <NUM> for the selected simulation scenarios. The information generated in this block of the pipeline <NUM> is added to the simulation scenarios as appropriate. Likewise, the planning/motion system simulation block <NUM> generates configuration information, state information and other information need to simulate the state and operation of the planning/motion system <NUM> based on outputs of the perception system simulation. This information is also added to the simulation scenarios. In some implementations, a simulation validation block <NUM> performs one or more steps to validate the additions to the simulation scenarios to ensure the simulation will execute properly when the simulation scenarios are used for execution by the simulator <NUM>. The pipeline <NUM> may also include other management or support features <NUM> needed by the simulator <NUM> to run simulations using the simulation scenarios. Finally, the simulation scenarios are output by the simulation data generator <NUM> for use by the simulator <NUM>. As noted above, in some implementations, the simulation data generator <NUM> stores the simulation scenarios as part of the simulation data <NUM> in the data storage.

Some example methods <NUM>, <NUM>, <NUM> of using the augmented data will now be described.

<FIG> is a flow chart illustrating a general method <NUM> of using the logged data to train a machine learning model <NUM> of an autonomous vehicle <NUM> in accordance with some implementations. In block <NUM>, logged data including sensor data is obtained from at least one sensor of an autonomous vehicle <NUM>. For example, the logged data may take the form of time-stamped logged data. The logged data may include, for example, localization data and tracking data that is time stamped, although more generally it may also include other vehicle data.

In block <NUM>, augmented data is generated based on the sensor data. This may include transforming the sensor data into a coordinate system of a simulation and performing data smoothing. This may, in some implementations, include generating metadata or performing formatting to facilitate access and used by other software entities.

In block <NUM>, simulation data, in particular one or more simulation scenarios, is generated based on the augmented data. Then simulations are executed using one or more simulation scenarios simulation to produce simulation data including simulation results and/or messages.

In block <NUM>, a machine learning model is trained based at least in part on the simulation data including simulation results and/or messages.

In block <NUM>, the trained machine learning model is applied to control an autonomous vehicle <NUM>. For example, during operation of the autonomous vehicle <NUM>, the trained machine learning model <NUM> may be used in a vehicle control system <NUM>.

<FIG> is a flowchart illustrating an implementation of a method <NUM> for utilizing the logged data <NUM> to generate simulation scenarios for simulations and generating data used to train a machine learning model <NUM> of an autonomous vehicle <NUM>. In block <NUM>, the time-series logged data is obtained from at least one sensor of an autonomous vehicle100. In block <NUM>, the time-series sensor data is converted into augmented data <NUM>. In block <NUM>, components of the augmented data are selected to generate a simulation scenario. For example, actors may be deleted or other components of the augmented data not selected, while other aspects of the data are varied. In block <NUM>, the selected components of the augmented data <NUM> are used to generate at least one simulation scenario for use in training the machine learning model <NUM>. The simulation scenario, in turn, may be used by one or more simulators <NUM> to generate a simulation result. In block <NUM>, the machine learning model <NUM> is trained based on a simulation result from a simulation that is based on at least one simulation scenario. In block <NUM>, the trained machine learning model <NUM> is applied to control the autonomous vehicle100.

<FIG> is a flow chart illustrating another example of a method <NUM> for using simulated data generated from logged data to train a machine learning model according to some implementations. In block <NUM>, time-series logged data is obtained for at least one autonomous vehicle <NUM>. The time-series logged data includes localization data and tracking data. One or more steps may be taken to transform the time-series logged data into a format that facilitates generating augmented data for use in a simulation. In block <NUM>, localization data and tracking data are mapped into global coordinates in order to map the motion of the ego-vehicle and actors in the global coordinate system. An example coordinate system is an orthogonal curvilinear coordinate (OCC) system. In one implementation, the actors are fit to a movement model to estimate their movement. In block <NUM>, at least one of data noise filtering, selection, and transformation is performed. In one implementation, the tracks are smoothed using a smoothing algorithm to make the smoothed tracks closer to ground level truth to form augmented data. In block <NUM>, the augmented data is used to generate a simulated perception system or a simulated plan for the autonomous vehicle <NUM>. The input may, in some implementations, be a simulation scenario and execution of the simulation scenario to generate a state or condition for the perception subsystem <NUM> or the planning subsystem <NUM>. However, more generally the input may include simulated sensor data. In block <NUM>, simulation data is generated by executing the simulation scenario. In block <NUM>, a machine learning model <NUM> of the autonomous vehicle <NUM> is trained based at least in part on the simulation data. In block <NUM>, the trained machine learning model <NUM> is applied to control the autonomous vehicle <NUM>.

The previous description is provided to enable practice of the various aspects described herein. Various modifications to these aspects will be understood, and the generic principles defined herein may be applied to other aspects. Thus, the claims are not intended to be limited to the aspects shown herein, but is to be accorded the full scope consistent with the language claims, wherein reference to an element in the singular is not intended to mean "one and only one" unless specifically so stated, but rather "one or more. All structural and functional equivalents to the elements of the various aspects described throughout the previous description that are known or later come to be known are expressly incorporated herein by reference and are intended to be encompassed by the claims. No claim element is to be construed as a means plus function unless the element is expressly recited using the phrase "means for.

It is understood that the specific order or hierarchy of blocks in the processes disclosed is an example of illustrative approaches. Based upon design preferences, it is understood that the specific order or hierarchy of blocks in the processes may be rearranged while remaining within the scope of the previous description.

The previous description of the disclosed implementations is provided to enable others to make or use the disclosed subject matter. Various modifications to these implementations will be readily apparent, and the generic principles defined herein may be applied to other implementations without departing from scope of the previous description. Thus, the previous description is not intended to be limited to the implementations shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The various examples illustrated and described are provided merely as examples to illustrate various features of the claims. However, features shown and described with respect to any given example are not necessarily limited to the associated example and may be used or combined with other examples that are shown and described. Further, the claims are not intended to be limited by any one example.

The foregoing method descriptions and the process flow diagrams are provided merely as illustrative examples and are not intended to require or imply that the blocks of various examples must be performed in the order presented. As will be appreciated, the order of blocks in the foregoing examples may be performed in any order. Words such as "thereafter," "then," "next," etc. are not intended to limit the order of the blocks; these words are simply used to guide the reader through the description of the methods.

The various illustrative logical blocks, modules, circuits, and algorithm blocks described in connection with the examples disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and blocks have been described above generally in terms of their functionality.

The hardware used to implement the various illustrative logics, logical blocks, modules, and circuits described in connection with the examples disclosed herein may be implemented or performed with a general purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. Alternatively, some blocks or methods may be performed by circuitry that is specific to a given function.

Claim 1:
A method for generating simulation data to be used in training a machine learning model of an autonomous vehicle (<NUM>), the method comprising:
receiving logged data (<NUM>) that includes sensor data obtained from at least one sensor (<NUM>) of the autonomous vehicle;
generating augmented data (<NUM>) from the logged data, the augmented data describing an actor (<NUM>; <NUM>; <NUM>) in an environment of the autonomous vehicle, the actor having an associated actor type and an actor motion behavior characteristic;
generating a simulation scenario as the simulation data (<NUM>), the simulation scenario generated from the augmented data;
executing a simulation based on the simulation scenario to generate a simulated output;
providing the simulation scenario as a training input to the machine learning model to generate a predicted output of the machine learning model; and
updating one or more weights in the machine learning model based on a difference between the predicted output and the simulated output of the simulation scenario.