Conditional agent trajectory prediction

Methods, computer systems, and apparatus, including computer programs encoded on computer storage media, for performing a conditional behavior prediction for one or more agents. The system obtains context data characterizing an environment. The context data includes data characterizing a plurality of agents, including a query agent and one or more target agents, in the environment at a current time point. The system further obtains data identifying a planned future trajectory for the query agent after the current time point, and for each target agent in the set, processes the context data and the data identifying the planned future trajectory using a first neural network to generate a conditional trajectory prediction output that defines a conditional probability distribution over possible future trajectories of the target agent after the current time point given that the query agent follows the planned future trajectory for the query agent after the current time point.

BACKGROUND

This specification relates to predicting the future trajectory of an agent in an environment.

The environment may be a real-world environment, and the agent may be, e.g., a vehicle in the environment. Predicting the future trajectories of agents is a task required for motion planning, e.g., by an autonomous vehicle.

Autonomous vehicles include self-driving cars, boats, and aircraft. Autonomous vehicles use a variety of onboard sensors and computer systems to detect nearby objects and use such detections to make control and navigation decisions.

Some autonomous vehicles have onboard computer systems that implement neural networks, other types of machine learning models, or both for various prediction tasks, e.g., object classification within images. For example, a neural network can be used to determine that an image captured by an onboard camera is likely to be an image of a nearby car. Neural networks, or for brevity, networks, are machine learning models that employ multiple layers of operations to predict one or more outputs from one or more inputs. Neural networks typically include one or more hidden layers situated between an input layer and an output layer. The output of each layer is used as input to another layer in the network, e.g., the next hidden layer or the output layer.

Each layer of a neural network specifies one or more transformation operations to be performed on the input to the layer. Some neural network layers have operations that are referred to as neurons. Each neuron receives one or more inputs and generates an output that is received by another neural network layer. Often, each neuron receives inputs from other neurons, and each neuron provides an output to one or more other neurons.

An architecture of a neural network specifies what layers are included in the network and their properties, as well as how the neurons of each layer of the network are connected. In other words, the architecture specifies which layers provide their output as input to which other layers and how the output is provided.

The transformation operations of each layer are performed by computers having installed software modules that implement the transformation operations. Thus, a layer being described as performing operations means that the computers implementing the transformation operations of the layer perform the operations.

Each layer generates one or more outputs using the current values of a set of parameters for the layer. Training the neural network thus involves continually performing a forward pass on the input, computing gradient values, and updating the current values for the set of parameters for each layer using the computed gradient values, e.g., using gradient descent. Once a neural network is trained, the final set of parameter values can be used to make predictions in a production system.

SUMMARY

This specification describes a system implemented as computer programs on one or more computers in one or more locations that generates a conditional trajectory prediction for a target agent, e.g., a vehicle, a cyclist, or a pedestrian, in an environment. The trajectory prediction is referred to as a “conditional” trajectory prediction because the system makes the prediction conditioned on a planned future trajectory for a query agent in the environment. In other words, the conditional trajectory prediction for the target agent is a prediction of the future trajectory of the target agent starting from a current time point given that or, in other words, assuming that the query agent will follow the planned future trajectory starting from the current time point.

Each conditional behavior prediction generated by the system defines a conditional probability distribution over a space of possible future trajectories for the target agent given that the query agent follows the planned future trajectory for the query agent starting from the current time point.

In some implementations, the system also generates a marginal probability distribution over the space of possible future trajectories for the target agent. The prediction of the marginal probability distribution is not conditioned on any planned trajectory for the query agent.

In these implementations, the system can use these two distributions to generate an interactivity score between the target agent and the query agent. When the environment includes multiple target agents, the system can generate a respective interactivity score between the query agent and each of the target agents.

The subject matter described in this specification can be implemented in particular implementations so as to realize one or more advantages.

When controlling an agent in an environment in which the agent potentially interacts with other agents, it is important to accurately model the reactions of other agents in the environment. For example, interactive driving scenarios, such as lane changes, merges, and unprotected turns, are challenging situations for autonomous driving. Planning in interactive scenarios requires accurately modeling the reactions of other agents, e.g., vehicles, cyclists, or pedestrians, in the environment according to different future actions of the autonomous vehicle to be controlled (i.e., the query agent).

The techniques described in this specification provide models for conditional trajectory prediction of the other agents that take as an input a query future trajectory for the query agent, and predict distributions over future trajectories for other agents conditioned on the query. Leveraging such models, the system can provide an agent interactivity score that characterizes the degree to which the query agent's behavior changes the predicted distribution of the corresponding target agent, and thus allowing identifying significant interactive scenarios for training and evaluating behavior prediction models. In real-world autonomous driving, the interactivity score can be used to anticipate driver interactions. When processing data offline, the interactivity scores can be used to mine interactive scenarios, for example, by identifying and focusing on scenarios having high interactivity scores.

Further, the interactivity scores are effective for guiding agent prioritization under computational budget constraints. For example, the system can make high-fidelity (and compute-intensive) behavior predictions only for target agents that have high interactivity scores. Because the interactivity scores are representative of likelihoods that the behavior of the query agent (e.g., the autonomous vehicle) will affect the corresponding agents, the system can reduce the amount of computational resources that are consumed by generating behavior predictions for surrounding agents without adversely impacting the quality of the final planned trajectory that is generated using the generated behavior predictions. In fact, in some cases, by causing the system to focus the high-fidelity predictions on only the relevant target agents, the quality of the final planned trajectory can be improved.

DETAILED DESCRIPTION

FIG.1Ashows an example of a conditional behavior prediction system100. The system100is an example of a system implemented as computer programs on one or more computers in one or more locations, in which the systems, components, and techniques described below can be implemented. For example, the system100can be deployed onboard an autonomous vehicle for generating predictions that guide the control of the autonomous vehicle. In an alternative example, the system100can be implemented on computers not onboard the autonomous vehicle to perform off-line predictions.

In general, the system100obtains context data110characterizing an environment. The context data includes data characterizing a plurality of agents, including a query agent112A and one or more target agents112B, in the environment at a current time point. In particular, the context data includes data characterizing trajectories of each of the plurality of agents up to the current time point.

The system100further obtains planned trajectory data120identifying a planned future trajectory for the query agent112A after the current time point.

For each target agent112B, the system100processes the context data110and the planned trajectory data120using a trajectory prediction neural network (130) to generate a conditional trajectory prediction140that defines a conditional probability distribution over possible future trajectories of the target agent after the current time point given that the query agent follows the planned future trajectory for the query agent after the current time point. The system can generate the output data180that includes the conditional trajectory prediction140and/or data derived from the conditional trajectory prediction140.

The trajectory prediction for the target agent112B is referred to as a “conditional” trajectory prediction because the system100makes the prediction conditioned on a planned future trajectory for the query agent112A in the environment. In other words, the conditional trajectory prediction for the target agent is a prediction of the future trajectory of the target agent112B starting from a current time point given that or, in other words, assuming that the query agent112A will follow the planned future trajectory starting from the current time point.

For example, the conditional trajectory prediction may be made by an onboard computer system of an autonomous vehicle navigating through the environment, the query agent112A may be the autonomous vehicle, and the target agent112B may be another vehicle, a pedestrian, or a cyclist that has been detected by the sensors of the autonomous vehicle. In these cases, the context data110includes data generated from data captured by one or more sensors, such as cameras and/or Lidar sensors, of the autonomous vehicle. The conditional behavior predictions can then be used by the onboard system to control the autonomous vehicle112A, i.e., to plan the future motion of the vehicle based in part on the likely consequences of one or more planned future trajectories on the motion of other agents in the environment. In particular, the system100can modify a future trajectory planned for the autonomous vehicle based on the conditional trajectory prediction, for example, to avoid an undesirable behavior (e.g., an abrupt braking, speeding up, crowding, or a fast turn) predicted for one or more other vehicles in the environment.

As another example, the conditional trajectory prediction may be made in a computer simulation of a real-world environment being navigated through by a simulated autonomous vehicle and the target agent, so that the query agent112A is the simulated autonomous vehicle and the target agent112B is another simulated vehicle in the vicinity of the simulated autonomous vehicle in the computer simulation. In this case, the context data includes data generated from data that simulates data that would be captured by one or more sensors of an autonomous vehicle in the real-world environment. Generating the predictions in simulation may assist in controlling the simulated vehicle, in testing the realism of certain situations encountered in the simulation, and in ensuring that the simulation includes surprising interactions that are likely to be encountered in the real world. More generally, generating these predictions in simulation can be part of testing the control software of a real-world autonomous vehicle before the software is deployed onboard the autonomous vehicle, of training one or more machine learning models that will layer be deployed onboard the autonomous vehicle or both.

As used in this specification, a future trajectory for an agent is a sequence that includes a respective agent state for the agent for each of a plurality of future time points, i.e., time points that are after the current time point. Each agent state identifies at least a waypoint location for the corresponding time point, i.e., identifies a location of the agent at the corresponding time point. In some implementations, each agent state also includes other information about the state of the agent at the corresponding time point, e.g., the predicted heading of the agent at the corresponding time point.

Each conditional behavior prediction generated by the system defines a conditional probability distribution over a space of possible future trajectories for the target agent given that the query agent follows the planned future trajectory for the query agent starting from the current time point.

In some implementations, the data characterizing trajectories of the plurality of agents can include the past states of the agents, including, for example, (x, y, z) positions and one or more of velocity vectors, acceleration vectors, orientation angles, and/or angular velocities. In some implementations, for each vehicle, the data can also include binary flags indicating features of the vehicle, e.g., whether the vehicle is signaling to turn left, signaling to turn right, and whether it is parked.

In some implementations, the context data110further includes road network information of the environment, including, for example, lane markings, lane boundaries, stop lines, and/or traffic light information, and so on. In one example, the context data includes points of lane markings and boundaries sampled around each agent.

To generate the conditional trajectory prediction140for each target agent112B, the system processes the context data110and the planned future trajectory data120using a trajectory prediction neural network130(referred to as a first neural network in this specification for convenience), to generate data that defines a conditional probability distribution over possible future trajectories of the target agent after the current time point given that the query agent follows the planned future trajectory for the query agent after the current time point.

In some implementations, the conditional trajectory prediction140includes, for each target agent in the set, a respective score for each possible future trajectory in a discrete set of possible future trajectories that represents a likelihood that the possible future trajectory will be followed by the target agent given that the query agent follows the planned future trajectory for the query agent. Each possible future trajectory in the discrete set can identify a respective future waypoint location of the target agent at each of a plurality of future time steps. In some of these implementations and for each target agent in the set, the conditional trajectory prediction output can further include, for each of the possible future trajectories and for each of the plurality of future time steps, parameters of an uncertainty distribution over the future waypoint location of the target agent at the future time step given that the target agent takes the possible future trajectory starting from the current time point.

In some implementations, the first neural network130includes a context data encoder neural network132, a trajectory encoder neural network134, and a decoder neural network136. The context data encoder neural network132is configured to, for each target agent112B, process the context data to generate an embedding of the context data for the target agent. The trajectory encoder neural network134is configured to, for each target agent112B, process the planned future trajectory to generate an embedding of the planned future trajectory of the query agent. The decoder neural network136is configured to, for each target agent, process (i) the embedding of the context data for the target agent and (ii) the embedding of the planned future trajectory for the target agent to generate the conditional trajectory prediction for the target agent.

An example of the first neural network130is described in detail with reference toFIG.1C. In general, the trajectory encoder neural network134is configured to, for each target agent, transform the planned future trajectory for the query agent into an ego-centric coordinate system with respect to the target agent and process the transformed planned trajectory to generate the embedding of the planned future trajectory. The context data encoder neural network132can be configured to, for each target agent, transform the context data into an ego-centric coordinate system with respect to the target agent and process the transformed context data to generate the embedding of the context data.

In some implementations, the first neural network130further includes a graph neural network138. The graph neural network is configured to receive features of a graph that has nodes representing the target agents and edges representing relationships between the target agents, and processes the features to generate a respective updated embedding for each of the target agents. The features of the input graph include, for each node in the graph, the embedding of the context data for the target agent represented by the node, the embedding of the planned trajectory for the target agent represented by the node, and the conditional trajectory prediction output for the target agent represented by the node. In this configuration, the first neural network is configured to process the respective updated embeddings of each of the target agents using the decoder neural network to update the conditional trajectory predictions for the target agents. In some implementations, for each target agent, the node representing the target agent is connected by a respective edge to the nodes representing the other target agents that are closest to the target agent in the environment at the current time point. In some implementations, the system100includes a fixed number (e.g., N=5) of neighboring agents that are nearest to the particular target agent in the graph representation. The system can process the features to generate a respective updated embedding for each of the target agents by computing edge features for each edge in the graph from at least the features of the nodes connected to the edge, and applying an attention-based aggregation function to combine the edge features.

In one example, a future trajectory s for an agent is defined as s={s1, . . . , sT}, which is a time discretized sequence of states for future time points t E {1, . . . , T}. The planned future trajectory for the query agent (e.g., agent A) is denoted by sA. sAcan be a specific realization of the distribution of future trajectories SAfor agent A. The trajectory prediction neural network receives as input a realization of the future trajectory of the query agent, sA=[s1A, . . . , sTA], and generates as the conditional trajectory prediction output that defines p(ŜB|SA=sA, x), the distribution of future trajectories for agent B conditioned on sA. Here, x represents observations from the environment, including past trajectories of all agents, and context information such as lane semantics.

In some implementations, the conditional trajectory prediction output includes a set of K trajectories for agent B, {circumflex over (μ)}B={{circumflex over (μ)}Bk}k=1K, where each trajectory is a sequence of states {circumflex over (μ)}Bk={{circumflex over (μ)}1Bk, . . . , {circumflex over (μ)}TBk}, capturing K potentially-different intended trajectories for agent B.

In some implementations, the system can predict uncertainty over the K intended trajectories as a softmax distribution {circumflex over (π)}Bk(x, sA). The model can also predict Gaussian uncertainty over the positions of trajectory waypoints as:
{circumflex over (ϕ)}Bk<(ŝtB|x,sA)=(ŝtB|{circumflex over (μ)}tBk(x,sA),{circumflex over (Σ)}tBk(x,sA)).  (1)

The system can generate the full conditional distribution p(ŜB|sA,x) as a Gaussian Mixture Model (GMM) with mixture weights fixed over all time steps of the same trajectory:
p(ŜB=ŝB|x,sA)=Σk=1K{circumflex over (π)}Bk(x,sA)Πt=1T{circumflex over (ϕ)}Bk(ŝtB|X,sA).  (2)

The softmax distribution can be computed as

π^B⁢k⁡(x,sA)=exp⁢f^kB⁡(x,sA)∑i⁢exp⁢f^iB⁡(x,sA).
The system can predict the parameters {circumflex over (f)}kB, {circumflex over (μ)}tBkand ΣtBksing a neural network, e.g., the first neural network.

In some implementations, the system100also generates a marginal trajectory prediction150for each target agent. Concretely, for each of the one or more target agents112B, the system100processes the context data110using a neural network (referred to as a second neural network in this specification for convenience) to generate a marginal trajectory prediction output that defines a marginal probability distribution over possible future trajectories of the target112B after the current time point without conditioning on the planned future trajectory for the query agent.

In some implementations, the system uses the same neural network that generates the conditional trajectory predictions, i.e., the first neural network130, to generate the marginal trajectory prediction. That is, the second neural network is the same neural network as the first neural network. By using the same trained neural network to produce both the conditional and the marginal predictions, the system100can potentially reduce the impact of training noise on the differences between marginal and conditional predictions.

To generate the marginal trajectory prediction150, the system100can process the context data and a pre-determined, default planned trajectory using the first neural network130to generate the marginal trajectory prediction150. For example, the first neural network150can include the context data encoder neural network152, the trajectory encoder neural network154, the decoder neural network156, and the graph neural network160, and the system can set an output of the planned trajectory encoder to a pre-determined, default embedding during the processing of the context data.

In one example, the system estimates the marginal probability distribution p(SB|x) for agent B, where p(SB|x) is not conditioned on any future plan for A. The marginal probability prediction can be computed based on parameters {tilde over (f)}kB(x), μtBk(x), and ΣtBk(x) generated by a neural network, e.g., the second neural network. Given intent, the distribution over agent B's future trajectories {tilde over (s)}tBcan be represented as
{circumflex over (ϕ)}Bk({tilde over (s)}tB|x)=({tilde over (s)}tB|{tilde over (μ)}tBk(x),{tilde over (Σ)}tBk(x)).  (3)

The full marginal distribution p(SB|x) over all potential intents for the agent B is given by
p({tilde over (S)}B={tilde over (s)}B|x)=Σk=1K{tilde over (π)}Bk(x)Πt=1T{circumflex over (ϕ)}Bk(ŝtB|x),  (4)
where

The system100can further include an interactivity characterization engine160that determines a respective interactivity score170for each target agent112B. In general, the interactivity score characterizes the degree to which the query agent's behavior changes the predicted distribution of the corresponding target agent.

In some implementations, the system100can use the interactivity score to guide the trajectory planning of the query agent, for example, by allocating computational resources for controlling the query agent starting from the current time point based on the respective interactivity scores for the target agents.

For example, when the environment includes multiple target agents, the system100can generate a respective interactivity score between the query agent and each of the target agents and can then allocate computational resources available for planning the future trajectory of the query agent, e.g., computational resources available onboard the autonomous vehicle, based on the interactivity scores.

For example, the system100can make high-fidelity (and compute-intensive) behavior predictions only for target agents that have interactivity scores above a threshold or only for the target agents that have the top k interactivity scores. The system can then use a lower-fidelity trajectory prediction system for the remaining agents.

Optionally, the system100can combine the interactivity scores with other factors, e.g., outputs of other models or heuristics such as distance, to generate a final interactivity score and then allocate resources based on the final scores.

Because the interactivity scores170are representative of likelihoods that the behavior of the query agent (e.g., the autonomous vehicle) will affect the corresponding agents, the system can reduce the amount of computational resources that are consumed by generating behavior predictions for surrounding agents without adversely impacting the quality of the final planned trajectory that is generated using the generated behavior predictions. In fact, in some cases, by causing the system to focus the high-fidelity predictions on only the relevant target agents, the quality of the final planned trajectory can be improved.

In general, the conditional trajectory prediction140for the one or more target agents112B and/or data derived from the conditional trajectory prediction outputs (e.g., the interactivity scores170) can be used to guide the control of the query agent. For example, when the query agent112A is an autonomous vehicle, the system100can provide (i) the conditional trajectory prediction outputs for the one or more target agents, (ii) data derived from the conditional trajectory prediction outputs for the one or more target agents, or (iii) both to an onboard system of the autonomous vehicle for use in controlling the autonomous vehicle. Similarly, when the query agent is a simulated autonomous vehicle in a computer simulation, the system100can provide (i) the conditional trajectory prediction outputs for the one or more target agents, and/or (ii) data derived from the conditional trajectory prediction outputs for the one or more target agents, for use in controlling the simulated autonomous vehicle in the computer simulation.

In some implementations, to determine the interactivity score170, the interactivity characterization engine160determines an estimate of a divergence between the conditional probability distribution140and the marginal probability distribution150predicted for each target agent112B.

In some implementations, to determine the estimate of the divergence, the interactivity characterization engine160samples a plurality of trajectories from the conditional probability distribution. The interactivity characterization engine160then determines, for each sampled trajectory, a ratio of (i) a probability assigned to the sampled trajectory by the conditional probability distribution to (ii) a probability assigned to the sampled trajectory by the marginal probability distribution. The system then determines the estimate from the ratios.

In some implementations, the interactivity characterization engine160adopts the Kullback-Leibler (KL) divergence (also termed the relative entropy), to compute the estimated divergence between the conditional and marginal distribution for the target agent's predicted future trajectory SBto quantify the influence exerted on agent B by a particular trajectory sA:

FIG.1Billustrates an example of an interactivity scenario between a query agent112A and a target agent112B. If the query agent112A (e.g., an autonomous vehicle driving on a road) is controlled to change lanes abruptly in front of the target agent112B (e.g., another vehicle sharing the road with112A), the target agent112B will likely change behavior (e.g., by slowing down), as shown by the conditional trajectory distribution. In this case, the KL-divergence will reflect a significant change relative to the agent's marginal expected behavior in the target agent's expected behavior as a result of the query agent's planned lane change.

Referring back toFIG.1A, the interactivity characterization engine160determines the interactivity score from at least the estimate of the divergence between the conditional probability distribution and the marginal probability distribution for each target agent.

In some implementations, to determine the respective interactivity score between the query agent112A and a particular target agent112B in the absence of a particular plan for the query agent or the target agent, the interactivity characterization engine160can take into account a set of possible trajectories for the query agent and compute the estimated divergence over all those possible trajectories. That is, the interactivity characterization engine160obtains (i) a plurality of possible planned future trajectories for the query agent that includes the planned future trajectory and (ii) a respective probability of occurrence for each of the plurality of planned future trajectories. The interactivity characterization engine160then determines, for each of the plurality of possible planned future trajectories, a respective estimate of a divergence between (i) a conditional probability distribution for the target agent conditioned on the possible planned future trajectory and (ii) the nominal probability distribution for the target agent. The system then determines the interactivity score from the probabilities for the plurality of possible planned future trajectories and the estimates for the plurality of possible planned future trajectories.

In some implementations, to obtain the plurality of possible planned future trajectories for the query agent112A and a respective probability of occurrence for each planned future trajectory, the interactivity characterization engine160uses the second neural network to generate the nominal trajectory prediction output for the query agent112A, and determines the plurality of possible planned future trajectories for the query agent112A and a respective probability of occurrence for each of the plurality of planned future trajectories using the nominal trajectory prediction output.

In one example, the possible future trajectories of agents A and B are represented by variables SAand SB. The system computes the interactivity score of the two agents' future trajectories SAand SBas:
I(SA,SB)=∫sAp(sA)DKL[P(SB|sA)∥p(SB)]  (6)

The interactivity score represents the dependence between the two random variables SAand SB. It is non-negative, I(SA,SB)≥0, and symmetric, I(SA,SB)=I(SB,SA).

In some implementations, the system approximates the marginal distributions by sampling N trajectories from the marginal distribution, {tilde over (s)}nA˜p({tilde over (S)}A|x), and computes the interactivity score as:

The system can estimate the KL divergence between the marginal and conditional GMM distributions via Monte-Carlo sampling of ŝmB˜p(ŜB|{tilde over (s)}nA,x), as:

This estimate needs to be evaluated for each {tilde over (s)}nA, requiring N inference steps of the conditional model. Instead, in some implementations, the system can estimate the outer expectation via importance sampling. That is, rather than sampling N samples from the marginal distribution, the system can use the K modes of the marginal distribution's GMM in Eq. (4), with {tilde over (s)}kA∈{{circumflex over (μ)}kA}k=1K:

I(SA,SB⁢x)≈1M⁢∑k⁢∑m⁢p(s˜kA⁢x)⁢log⁢p(S^B=s^mB⁢s˜kA,x)p(S~B=s^mB⁢x)(9)
where the marginal and conditional probabilities are evaluated via Eqs. (4) and (2).

In order to cause the neural network130to make accurate trajectory predictions, the system100or another system trains the trajectory prediction neural network130using training data including a plurality of training examples. For example, for predicting trajectories of agents (e.g., vehicles, pedestrians, and cyclists) in an interactive driving environment, the system can use datasets of logged driving data to train the trajectory prediction neural network via end-to-end supervised learning. For example, the logged driving data can include driving sequences of autonomous vehicles.

In one example, to train the model for conditional prediction, the system can set the conditional query/plan input to agent A's ground-truth future trajectory from the training dataset. The system can further train the model for marginal predictions by setting the conditional query/plan to a zero vector sA=0. Therefore, marginal prediction can be obtained via {tilde over (f)}kB(x):={circumflex over (f)}kB(x, 0), {tilde over (μ)}tBk(x):=μtBk(x, 0), and {tilde over (Σ)}tBk(x):={circumflex over (Σ)}tBk(x, 0). During training, the system can select a portion of (e.g., 95%) of examples to include conditional queries and use the other (e.g., 5%) of examples as marginal query examples.

The system can train the trajectory prediction neural network to predict the distribution parameters {circumflex over (f)}kB(x, sA), {circumflex over (μ)}tBk(x, sA), and ΣtBk(x, 0) (x, sA) via supervised learning using the negative log-likelihood loss,

In some implementations, instead of producing predictions for a single target agent B, the trajectory prediction neural network is trained to generate predictions for multiple target agents in parallel. This increases the computational efficiency for the system to make trajectory predictions for multiple target agents in the environment. In particular, the conditional trajectory prediction model can be configured to process data for each of the multiple agents in parallel in a batch dimension. For a particular target agent B, the system can transform the scene context into a coordinate system centered on the target agent B, and predict agent B's future trajectory distributions as a set of trajectories in agent B's coordinate system. The system can further transform the output trajectories for the multiple target agents back into the global coordinate system for the environment. In order to take into account the fundamental physical property that the agents cannot occupy the same future location in space-time, the system can include an additional loss term,
(A,B)=ΣΣ{circumflex over (π)}Ai{circumflex over (π)}Bjmaxt(∥{circumflex over (μ)}tAi−{circumflex over (μ)}tBj∥22<α),  (11)
where {({circumflex over (π)}Ai, {circumflex over (μ)}Ai)}i=1Kand {({circumflex over (π)}Bj, {circumflex over (μ)}Bj)}j=1Kare the modes and probabilities of the future trajectory distributions for agents A and B. In some implementations, the constraint, ∥⋅∥<α, can be relaxed to obtain a differentiable loss term:

FIG.1Cshows an example of the trajectory prediction neural network130. The trajectory prediction neural network130includes the query encoder134for encoding the query agent's planned future trajectory for each target agent and a decoder neural network that includes the graph neural network138for updating embeddings for each target agent. The context data encoder can include a road graph encoder132A for encoding road features and an agent interaction encoder132B for encoding agent past trajectories.

In one example, the road graph encoder132A samples road features (such as lanes, traffic lights, stop lines) at specified intervals (e.g., at 1 m intervals) around each agent, to obtain sampled points for these features. The road graph encoder can transform these points into each agent's reference frame, and processes them via a neural network, such as a multi-layer perceptron (MLP). The road graph encoder132A can then aggregate the variable number of points via max-pooling to obtain a fixed size encoded vector to describe the road graph around each agent.

In one example, the agent interaction encoder132B receives a sequence of past state observations (e.g., for a 2 s period until a current time point sampled at 10 Hz). The agent interaction encoder132B can encode the time series of state vectors with a recurrent neural network (e.g., an LSTM). The agent interaction encoder132B can take into account agent interactions by treating each agent as an ego-agent. That is, for each particular agent, the encoder132B can transform the state vector sequences of the other agents into the coordinate frame of the particular agent. The encoder can then encode the transformed sequences, e.g., via an MLP. The encoder132B can then aggregate the neighbor encodings with a max-pool operation to obtain a single encoding summarizing all interactions with neighboring agents.

Similarly, in one example of the query encoder134, the query encoder134can encode the conditional query trajectory using an MLP. The query is encoded for each of the target agents in the scene, in their respective coordinate frame.

The system can combine (e.g., concatenate) the encoded embeddings outputted from the encoders132A,132B, and134, and decode the combined embeddings into trajectories and likelihoods using the decoder neural network136. In some implementations, the decoding process can be performed iteratively by the graph neural network138.

In one example, the system computes trajectory likelihood scores using a classification head136A (e.g., a neural network with a fully-connected ResNet architecture) followed by a softmax output. The system further regresses the trajectory coordinates of the predicted trajectories into a polynomial using an intent trajectory head136B (e.g., a neural network having ResNet blocks for each of the x and y spatial dimensions).

The graph neural network138can receive the embeddings, the decoded trajectories, and the decoded likelihoods for each agent as node features. An example of the graph neural network architecture is the SpaGNN neural network described in “Spatially-aware graph neural networks for relational behavior forecasting from sensor data,” arXiv: 1910.08233, 2019. The input graph can be constructed using every predicted agent as a node in the graph, and each node is connected to those corresponding neighboring agents (e.g., the five nearest agents). The graph neural network138can use an attention-based aggregation function that combines the edge features to form messages passed to each node.

In each of a plurality of iterations in the decoding process, the system can apply one message update in the graph neural network to pass trajectory information between neighboring agents, and then apply trajectory decoding. This process can refine the agents' trajectory distributions with awareness of their neighbors' distributions.

FIG.2Ais a flow diagram illustrating an example process200for performing a conditional behavior prediction. For convenience, the process200will be described as being performed by a system of one or more computers located in one or more locations. For example, a conditional behavior prediction system, e.g., the conditional behavior prediction system100ofFIG.1A, appropriately programmed in accordance with this specification, can perform the process200.

In step210, the system obtains context data characterizing the environment. The context data includes data characterizing a plurality of agents, including a query agent and a set of one or more target agents, in the environment at a current time point. The context data includes data characterizing trajectories of each of the plurality of agents up to the current time point.

In one example, the query agent is an autonomous vehicle navigating through the environment. Each target agent in the set is an agent in a vicinity of the autonomous vehicle in the environment. The context data includes data generated from data captured by one or more sensors, such as cameras and/or Lidar sensors, of the autonomous vehicle.

In another example, the query agent is a simulated autonomous vehicle navigating through a computer simulation of a real-world environment. Each target agent in the set is a simulated agent in a vicinity of the simulated autonomous vehicle in the computer simulation. The context data includes data generated from data that simulates data that would be captured by one or more sensors of an autonomous vehicle in the real-world environment.

In some implementations, the data characterizing trajectories of the plurality of agents can include the past states of the agents, including, for example, (x, y, z) positions, and one or more of, velocity vectors, acceleration vectors, orientation angles, and/or angular velocities. In some implementations, for each vehicle, the data can also include binary flags indicating features such as whether the vehicle is signaling to turn left, signaling to turn right, and whether it is parked.

In some implementations, the context data further includes road network information of the environment, including, for example, lane markings, lane boundaries, stop lines, and/or traffic light information, and so on. In one example, the context data includes points of lane markings and boundaries sampled around each agent.

In step220, the system obtains planned trajectory data for the query agent. The planned trajectory data identifies a planned future trajectory for the query agent after the current time point.

For example, the query agent can be an autonomous vehicle navigating through the environment, and the planned future trajectory can be generated by the onboard system of the autonomous vehicle.

In step230, the system generates a conditional trajectory prediction output for each target agent. Concretely, for each target agent in the set, the system processes the context data and the data identifying the planned future trajectory using a trajectory prediction neural network (referred to as a first neural network in this specification for convenience), to generate a conditional trajectory prediction output that defines a conditional probability distribution over possible future trajectories of the target agent after the current time point given that the query agent follows the planned future trajectory for the query agent after the current time point.

In some implementations, the conditional trajectory prediction output includes, for each target agent in the set, a respective score for each possible future trajectory in a discrete set of possible future trajectories that represents a likelihood that the possible future trajectory will be followed by the target agent given that the query agent follows the planned future trajectory for the query agent. Each possible future trajectory in the discrete set can identify a respective future waypoint location of the target agent at each of a plurality of future time steps, and for each target agent in the set, the conditional trajectory prediction output can further include, for each of the possible future trajectories and for each of the plurality of future time steps, parameters of an uncertainty distribution over the future waypoint location of the target agent at the future time step given that the target agent takes the possible future trajectory starting from the current time point.

In some implementations, the first neural network includes a context data encoder neural network, a trajectory encoder neural network, and a decoder neural network. The context data encoder neural network is configured to, for each target agent, process the context data to generate an embedding of the context data for the target agent. The trajectory encoder neural network is configured to, for each target agent, process the planned future trajectory to generate an embedding of the planned future trajectory of the query agent. The decoder neural network is configured to, for each target agent, process (i) the embedding of the context data for the target agent and (ii) the embedding of the planned future trajectory for the target agent to generate the conditional trajectory prediction for the target agent.

In some implementations, the trajectory encoder neural network is configured to, for each target agent, transform the planned future trajectory into an ego-centric coordinate system with respect to the target agent and process the transformed planned trajectory to generate the embedding of the planned future trajectory. The context data encoder neural network can be configured to, for each target agent, transform the context data into an ego-centric coordinate system with respect to the target agent and process the transformed context data to generate the embedding of the context data.

In some implementations, the first neural network further includes a graph neural network. The graph neural network is configured to receive features of a graph that has nodes representing the target agents and edges representing relationships between the target agents, and processes the features to generate a respective updated embedding for each of the target agents. The features of the input graph include, for each node in the graph, the embedding of the context data for the target agent represented by the node, the embedding of the planned trajectory for the target agent represented by the node, and the conditional trajectory prediction output for the target agent represented by the node. In this configuration, the first neural network is configured to process the respective updated embeddings of each of the target agents using the decoder neural network to update the conditional trajectory predictions for the target agents. In some implementations, for each target agent, the node representing the target agent is connected by a respective edge to the nodes representing the other target agents that are closest to the target agent in the environment at the current time point. The system can process the features to generate a respective updated embedding for each of the target agents by computing edge features for each edge in the graph from at least the features of the nodes connected to the edge, and applying an attention-based aggregation function to combine the edge features.

Optionally, in step240, the system determines a respective interactivity score for each target agent. In general, the interactivity score characterizes the degree to which the query agent's behavior changes the predicted distribution of the corresponding target agent.

In some implementations, the system can use the interactivity score to guide the trajectory planning of the query agent, for example, by allocating computational resources for controlling the query agent starting from the current time point based on the respective interactivity scores for the target agents.

FIG.2Bis a flow diagram illustrating an example process240for determining the respective interactivity score for each target agent. For convenience, the process240will be described as being performed by a system of one or more computers located in one or more locations. For example, a conditional behavior prediction system, e.g., the conditional behavior prediction system100ofFIG.1A, appropriately programmed in accordance with this specification, can perform the process240.

In step242, the system generates a marginal trajectory prediction output for each target agent. Concretely, for each of the one or more target agents, the system processes the context data using a neural network (referred to as a second neural network in this specification for convenience) to generate a marginal trajectory prediction output that defines a marginal probability distribution over possible future trajectories of the target after the current time point without conditioning on the planned future trajectory for the query agent.

In some implementations, the system uses the same neural network that generates the conditional trajectory predictions, i.e., the first neural network, to generate the marginal trajectory prediction. That is, the second neural network is the same neural network as the first neural network. By using the same trained neural network to produce both the conditional and the marginal predictions, the system can potentially reduce the impact of training noise on the differences between marginal and conditional predictions.

To generate the marginal trajectory prediction output, the system can process the context data and a pre-determined, default planned trajectory using the first neural network to generate the marginal trajectory prediction output. For example, the first neural network can include the context data encoder neural network, the trajectory encoder neural network, the decoder neural network, and the graph neural network, and the system can set an output of the planned trajectory encoder to a pre-determined, default embedding during the processing of the context data.

In step244, the system determines an estimate of a divergence between the conditional probability distribution and the marginal probability distribution for each target agent.

In some implementations, to determine the estimate of the divergence, the system samples a plurality of trajectories from the conditional probability distribution. The system then determines, for each sampled trajectory, a ratio of (i) a probability assigned to the sampled trajectory by the conditional probability distribution to (ii) a probability assigned to the sampled trajectory by the marginal probability distribution. The system then determines the estimate from the ratios.

In step246, the system determines the interactivity score from at least the estimate of the divergence between the conditional probability distribution and the marginal probability distribution for each target agent.

In some implementations, to determine the respective interactivity score between the query agent and a specific target agent in the absence of a particular plan for the query agent or the target agent, the system can take into account a set of possible trajectories for the query agent and compute the estimated divergence over all those possible trajectories. That is, the system obtains (i) a plurality of possible planned future trajectories for the query agent that includes the planned future trajectory and (ii) a respective probability of occurrence for each of the plurality of planned future trajectories. The system then determines, for each of the plurality of possible planned future trajectories, a respective estimate of a divergence between (i) a conditional probability distribution for the target agent conditioned on the possible planned future trajectory and (ii) the nominal probability distribution for the target agent. The system then determines the interactivity score from the probabilities for the plurality of possible planned future trajectories and the estimates for the plurality of possible planned future trajectories.

In some implementations, to obtain the plurality of possible planned future trajectories for the query agent and a respective probability of occurrence for each planned future trajectory, the system uses the second neural network to generate a nominal trajectory prediction output for the query agent, and determines the plurality of possible planned future trajectories for the query agent and a respective probability of occurrence for each of the plurality of planned future trajectories using the nominal trajectory prediction output.