Is Anyone There? Learning a Planner Contingent on Perceptual Uncertainty
Abstract: Robots in complex multi-agent environments should reason about the intentions of observed and currently unobserved agents.
In this paper, we present a new learning-based method for prediction and planning in complex multi-agent environments where the states of the other agents are partially-observed.
Our approach, Active Visual Planning (AVP), uses high-dimensional observations to learn a flow-based generative model of multi-agent joint trajectories, including unobserved agents that may be revealed in the near future, depending on the robot’s actions.
Our predictive model is implemented using deep neural networks that map raw observations to future detection and pose trajectories and is learned entirely offline using a dataset of recorded observations (not ground-truth states).
Once learned, our predictive model can be used for contingency planning over the potential existence, intentions, and positions of unobserved agents.
We demonstrate the effectiveness of AVP on a set of autonomous driving environments inspired by real-world scenarios that require reasoning about the existence of other unobserved agents for safe and efficient driving.
In these environments, AVP achieves expertlevel closed-loop performance, while methods that do not reason about potential unobserved agents exhibit either overconfident or underconfident behavior.
Introduction
When navigating in complex multi-agent scenarios, humans reason about the uncertainty of their own perception and how their actions affect their perception [1].
For example, a human driver at a blind intersection depicted in Fig.
1a may inch forward to improve their visibility of oncoming traffic before deciding whether or not to proceed.
This type of explicit reasoning is a natural human behavior critical to safe and efficient decision making.
However, it does not exist in autonomous systems that assume full observability of the environment, nor in systems that assume their limited situational awareness is persistent and unactionable.
While it is possible to engineer human-like behavior for many individual scenarios involving perceptual uncertainty, can we instead develop a principled data-driven solution to learn end-to-end planners that incorporates perceptual uncertainty?
The partially observable Markov decision process (POMDP) model admits a notion of perceptual uncertainty, where the state of the underlying MDP is hidden due to imperfect perception (such as visual occlusions or sensor noise) and must be estimated by the agent in order to construct an optimal policy.
POMDPs can be solved exactly for small discrete state-action spaces, but currently cannot be in large or continuous state-action spaces typical of AV planning [13], leading to a variety of approximate solutions that require significant human design effort, e.g. state spaces, affordances, scenario-specific heuristics, and/or dynamics models.
Reinforcement learning (RL) can be applied to POMDP problems without requiring human-defined states or dynamics, however, RL adds a significant burden of online data collection.
We propose a tractable data-driven method to accommodate partial observability, unknown dynamics, raw observations, and offline data.
(a)
Planning
Learned Learned Rawobs
[Figure]
Contingency Closed
[Figure]
loop method model controller asinput observ.
inobserv.
env.evaluation Fisacetal.[2] ✗
None ✗ ✗
None ✓ Bajcsyetal.[3] ✓
Offline ✗ ✗
None ✓ Ivanovicetal.[4] ✓
Offline ✗ ✗
Active ✗ Cuietal.[5] ✓
Offline ✓ ✗
Active ✗ Rhinehartetal.[6] ✓
Offline ✓ ✗
Active ✓ Tasetal.[7] ✗
None ✗ ✓
None ✓ McGilletal.[8] ✗
None ✗ ✓
None ✓ Wangetal.[9] ✗
None ✗ ✓
Active ✓ Andersenetal.[10] ✗
None ✗ ✓
Active ✓ Zhangetal.[11] ✗
None ✗ ✓
Active ✓ Wrayetal.[12] ✗
None ✗ ✓
Active ✓ AVP( ours ) ✓
Offline ✓ ✓
Active ✓ (b)
In this paper, we present Active Visual Planning (AVP), a scalable method for contingency planning under multiple forms of partial observability (uncertainty in existence, positions, and intentions) that accommodates unknown dynamics, raw high-dimensional observations such as camera images or LiDAR scans, and learning from offline data.
AVP models discrete uncertainty (uncertainty in existence) and continuous uncertainty (uncertainty in position) by reasoning about the existence of other entities and their future trajectories and planning accordingly.
Importantly, AVP incentivizes an agent to take actions that actively reduce uncertainty over the existence of unobserved entities (e.g., moving to a location with a wider field of view) if it improves the quality of the overall plan (only a small subset of all uncertainty induced by imperfect perception is relevant to near-term decision making in many real-world scenarios).
We evaluate AVP in various multi-agent settings in the CARLA [14] simulation environment.
Our main contributions are:
• The first fully-learned planner that supports active contingency planning over perceptual uncertainty.
• A benchmark for evaluating the robustness of planners to real-world examples of perceptual uncertainty in a visually realistic simulator (CARLA).
We illustrate that our method outperforms methods that do not properly account for potential occluded vehicles.
Supplementary material can be found at https://sites.google.com/view/active-visual-planning.
Related Work
POMDPs and belief space planning.
A long history of research in POMDP solvers and belief space planning for autonomous driving exists [15, 16, 17], and recent work has demonstrated success in modern autonomous vehicle (AV) simulators and real-world systems [18, 19, 20, 21, 22, 23].
Simple POMDPs can be solved using brute force enumeration [24], incremental pruning [25], and linear support [26], but in large or continuous state-action spaces typical of AV planning, exact POMDP solutions are generally intractable [13], leading to approximations such as Point
Based Value Iteration [27, 28] and Heuristic Search Value Iteration [29], or making simplifying assumptions for tractability such as assuming belief distributions are unimodal, as iterative-linear (e.g. iLQG controllers) or moment-matching methods [30] do.
Sampling-based Monte Carlo tree search methods [31, 32, 33] improve on point-based solvers and enable scaling to larger POMDPs with continuous state and action spaces.
However, approximate methods that have been shown to work on real AV systems require human-defined state spaces, affordances, and scenario-specific heuristics [12] (e.g. augmenting states with affordances such as time since last stop, distance to stop sign, etc).
In contrast, our modeling framework (AVP) is data-driven and automatically learns both a dynamics model and contextual features from raw sensor observations, and we demonstrate that the same model architecture and planner can generalize to several different scenarios without modification.
We illustrate these differences in Fig.
1b.
Reinforcement and imitation learning.
Although the MDP framework of reinforcement learning (RL) assumes the agent receives state observations, POMDPs can be reduced to MDPs by recording the action-observation history (or a sufficient statistic of the history) [34], and in practice deep RL has been shown to work well in certain POMDP environments using truncated histories and recurrent neural networks [35].
Both model-free [36, 37, 38, 39] and model-based deep RL [40] have been successfully applied to AV tasks to plan directly from images.
While RL requires learning online , our approach is trained fully offline from logged data.
Recent work in offline RL [41] aims to use RL on previously collected data, but both offline and online RL require access to a reward function during training, whereas AVP does not.
AVP is closely related to imitation learning (IL), which also learns a control policy offline [42, 43].
However, unlike IL, AVP only requires expert observations instead of expert demonstrations (i.e. AVP does not require action labels) and is flexible to planning new goals at test-time, as opposed to IL which directly learns a control-level policy.
Occlusion-aware prediction and planning.
In settings with visual occlusions, map knowledge and geometry can be employed for occlusion-aware prediction [44, 45, 46] and planning [47, 48, 7, 8, 9, 10].
For example, forward reachable set (FRS) methods avoid the entire set of future reachable states of hypothetical vehicles placed on the edges of occluded regions.
However, such approaches often result in overly conservative plans, not only by assuming the worst case in existence and position, but also by assuming the worst case in intention (i.e., that entities are controlled by adversarial policies).
In contrast, AVP learns a probabilistic model of existence, positions, and intention, enabling our planner to weigh risk and reward instead of relying on worst-case analysis.
AVP also does not utilize any explicit map knowledge or geometry (such as estimated field of view), and instead learns to use raw sensor observations.
Game theory has been applied to occlusion-aware planning to improve on worst-case analysis by modeling interactions between vehicles [11].
Although game-theoretic planners are capable of generating expert-level driving policies, in practice, they are difficult to apply to scenarios with more than two vehicles, whereas AVP naturally scales beyond two agents.
Active Visual Planning
Preliminaries: We consider multi-agent settings comprising an unknown number of agents, which we model as a fixed-size large set of agents, with some potentially not present.
We will subsequently refer to the agent that is guaranteed to exist as the robot ego , for which we are designing a controller, and A other agents as human actors (no direct control).
We consider continuous positions in d -dimensional
Cartesian space and discrete time, and denote the position of agent
a at time t as x at ∈ R d , and omit the superscript to denote the joint positions of all agents as x t ∈ R A × d .
Let t = 0 denote the current timestep, x 0 denote the current joint positions, and x ≤ t denote future joint positions from 1 to t .
We say non-ego actors x 1: A t are observed if and only if the actor is detected by the ego.
We denote if an actor a has been detected by ego at time t with a binary flag d at , and the one-hot vector of all detections as d t ∈ R A ; d at = 1 if actor a has been observed in the ego agent’s perceptual context
i (e.g., LiDAR sensors or camera images).
˜ x t denotes an observed joint position from the ego’s perspective, where for each agent a ̸ = 0 , the true position is revealed ( ˜ x at = x at ) if d at = 1 , and is masked otherwise.
The robot’s current observation is perceptual context and a subset of the past joint trajectories, denoted o .
=
(˜ x ≤ 0 , i 0 ) .
Uppercase denotes random variables, lowercase denotes realized values: X ≤ t and D ≤ t represent a stochastic sequence of future positions and future detections.
A fully-observed model (prior work): We base our models on prior work that introduced an autoregressive probabilistic model for multi-agent prediction [49] and planning [6] with several desirable properties: the predictive model can be fully learned from observations using maximum likelihood estimation (no need to specify an approximate dynamics model), it supports multi-modal joint predictions over varying numbers of agents, models co-dependencies between different agents, and enables straightforward yet expressive planning through gradient-based optimization of the model.
Importantly, the plans generated through the model are contingent , i.e., the plans explicitly account for (are contingent on) different future outcomes of modelled stochastic events.
However, these modelled events did not include the potential appearance of new agents, which precludes explicitly reasoning about such events.
In many real-world driving scenarios, full observability is an unrealistic assumption: occlusions frequently block the view of other relevant agents, and the imperfect sensing and perception systems used in modern self-driving cars can often lead to low confidence or erroneous object detection and tracking.
Our model enables prediction and planning with respect to currently-unobserved agents.
The starting point for our behavioral model is a parameteric probabilistic model q θ ( X | o ) of future joint trajectories X ≤ t , conditioned on o (recall o is perceptual context i 0 and past trajectories x ≤ 0 ).
The distribution is factorized across both time ( 1 ...T ) and agents ( 0 ...A ) into a product of autoregressive normal distributions q aθ of present positions t for each agent a , and q aθ is conditioned on a paramaterized transformation of previous predictions and the context, denoted as s
[Equation] [Equation]
[Equation] [Equation]
<
[Equation]
, i ) :
[Equation]
where X can be sampled by drawing Z ∼ N (0 , I ) :
X at = µ θ ( s t − 1 ) + σ θ ( s t − 1 ) ·
Z at .
This particular parameterization allows us to express variation in position as a probabilistic noise component drawn from a simple base distribution Z ; for the ego vehicle, we assume this noise is controllable (and thus fix z 0 while planning), whereas for non-ego actors we sample z 1: A ∼ Z 1: A .
Modeling partial observability: To adapt the model in Eq.
1 to the partially observed setting where many entries of x are often unknown, we model joint trajectories ˜
x of detected agents indices
D .
= { ( t, a ) : d at = 1 } as q partial-obs.
θ , and model the joint probability of future agent detection as v θ : q partial-obs.
θ
[Equation]
where v at is a Bernoulli distribution predicted by a neural-network that represents the probability that agent a is detected at time t .
Note d is observed during training (even when x is not).
Since we no longer assume access to x , s t − 1 is instead a transformation of ˜ x .
Because v at is conditioned on the perceptual context, the model can relate visual cues (such as visibly occluded areas of a LiDAR point cloud or camera image) with the probability of future detection.
Benefits of modeling detection instead of existence : Eq.
(4) models currently undetected agents appearing into view, and also currently detected agents disappearing from view.
For example, this behavior is seen in the overtake and intersection environments in Fig.
3: if the ego agent successfully navigates past the other agent, the non-ego agent will eventually be predicted to “disappear” (become undetected) as it drives out of the ego vehicle’s LiDAR range.
By modeling detection instead of existence, we can train a partially-observed trajectory model without requiring privileged access to ground truth position information for all occluded agents; a learned model of existence similar to the model of detection in Eq.
(4) (via a Bernoulli variable) would require explicitly modeling the positions of agents while undetected.
AVP implicitly considers the dynamics between detected and undetected agents and retains aleatoric uncertainty in each distribution as a function of the observed
Require:
Trained predictive model M (consisting of a context encoder and autoregressive state encoders and decoder), partially observed past joint trajectories ˜ x ≤ 0 , present perceptual context i 0 , and planning loss criterion L 1 Randomly initialize sequence of robot
z ’s (each z r denoted as z 0 t ) 2 Compute initial state encoding s 0 using learned trajectory and context encoders on { ˜ x ≤ 0 , i 0 } 3 for each optimization step dofor each sample dofor each t in horizon T do
Pass the state encoding s t − 1 to the learned trajectory decoder to get D 1: A t , X 1: A t , and RNN hidden state h t
Sample z 1: A t ∼ N (0 ,I ) for every non-controllable (i.e., human) actor to compute x 1: A t (use fixed z 0 t to compute x 0 t )
Sample d 1: A t ∼ D 1: A t and combine with x 0:
[Table]
t to compute the observed positions ˜ x 0: A t
Append d
[Table]
and
[Table]
to
[Table]
to
[Table]
Compute the gradient of the joint
[Table]
∇L (
[Table]
z
[Table]
) using
[Table]
Use
[Table]
to
[Table]
information (perceptual context and positions of detected agents); see the Fig.
3 in the supplement for a visual explanation.
Planning with a learned model: To use our learned model for planning, we plan a sequence of z ’s for the controllable robot agent, while sampling z ’s for all other human-controlled agents.
By planning in z -space, we are effectively planning policies , or contingent plans: fixing z r creates a deterministic robot policy from a partial policy π r .
=
q 0 θ , which is specified by the learned parameters θ of the model.
Because z h remains sampled, the overall system remains stochastic; planning over z h erroneously implies that all agents are explicitly controllable.
Following [6], planning is derived as maximum a posteriori estimation of the free policy parameters z r ≤ T conditioned on the initial positions and perceptual context, as well as a set of high-level goals G =
( g , G ) consisting of target coordinate locations g and constraints G on the joint trajectory.
In our experiments we apply a minimum distance constraint (8 meters to the nearest vehicle), which corresponds to a near-collision penalty.
Estimation of posterior p ( z r ≤ T |G , o ) is a relatively lightweight gradient-based optimization procedure, since we optimize a small number of free parameters z r while keeping all other parameters of the learned model fixed.
For our cost function G , we adopt the same lower bound planning objective as [6], which balances (1) a behavioral prior, (2) a scaled distance to the destination, and (3) a set of indicator functions δ G ( x ) on the optional constraints:
[Equation]
Optimizing z r ≤ T according to Eq.
5 thus produces a policy that (with high expectation) reaches a destination specified at test-time, while mimicking human-like driving behavior seen in the expert observations.
The optimization of Eq.
5 accounts for the detection model because it is a function of ˜ x
(note that o decomposes to { ˜ x , d , i } ), which itself is a function of the agent detection model (see Eq.
See Algorithm 1 for pseudocode describing the planning procedure.
Model architecture and training
: We implement autoregressive models q θ and v θ using an encoderdecoder architecture (see Fig. 2) by building on the open-source implementation of [6].
f θ is implemented as two separate networks: a convolutional neural network (CNN) that encodes the perceptual context, and a recurrent neural network (RNN) trajectory encoder that reads a past trajectory and generates an encoding.
q θ is implemented as an RNN decoder that takes the encoding and generates a sequence of µ and σ representing future positions.
Similarly, v θ is implemented as an
RNN that generates a sequence of Bernoulli parameters p .
The entire network is jointly trained on the dataset of observations (consisting of observed joint trajectories and perceptual context) using maximum likelihood estimation according to Eqs.
(3) and (4).
During inference, if the v θ does not predict a detection ( d h ∼ v θ = 0 ), then the predicted position (sampled from q θ ) is ignored.
AVP can handle scenarios with varying numbers of agents as long as the model was trained on a dataset that includes the same number of agents, as shown by the three-agent intersection experiment.
Beyond this practical example of AVP’s efficacy, theoretically, an agent trained on a dataset containing up to
N agents will be able to handle up to N agents at test-time, and the variability in number of agents can be handled through the modeling of detection (the majority of N agents will then be predicted as undetected).
In our experiments, we use a CNN with three convolutional layers (each with 32 filters
[Figure]
and an 8 × 8 kernel) as our context encoders, and GRUs with three layers (256 hidden units each) as our recurrent encoders and decoders.
The initial observation (model input) during model training contains a 60 × 360 LiDAR range map and a partially observed trajectory with 15 waypoints, sampled at 20Hz.
The future trajectory (target sequence) consists of 30 waypoints, sampled at 7.5Hz.
Further details can be found in the supplemental material.
Experiments
Scenarios requiring decision-making under perceptual uncertainty are ubiquitous in the domain of autonomous vehicles: occlusions obscuring important objects and agents are unavoidable even with state-of-the-art LiDAR, RADAR, and image-based sensors, additionally, sensor performance can significantly degrade under certain conditions such as inclement weather.
To test the capability of our method, we focus on several autonomous driving scenarios where correctly reasoning about the existence of occluded agents leads to a clear difference in planner behavior when evaluated in a closed-loop control setting.
We implement these scenarios, visualized in Fig.
3, in the CARLA simulator [14].
Our key hypothesis is that in these scenarios, the ability of the AVP to predict and plan with respect to unobserved vehicles will result in both safe and efficient control compared to existing approaches that do not properly account for unobserved vehicles.
4.1 Scenarios
Two-lane overtake (a): The ego vehicle attempts to overtake a truck on two-lane freeway and must avoid potential oncoming traffic.
An agent that is correctly planning with respect to hypothetical oncoming traffic will make the lane change quickly to avoid getting stuck parallel to the truck with no escape maneuver.
An overconfident agent that is oblivious to the possibility of occluded oncoming traffic will attempt the overtake maneuver at a lower speed, opening up the possibility of an unavoidable collision.
An underconfident agent will never initiate an overtake.
Blind summit (b): The ego vehicle approaches a steep incline occluding the road past the apex of the hill.
Because another vehicle may exist at the top of the hill, an expert agent will slow down just enough to avoid an unavoidable collision with a hypothetical occluded vehicle.
An overconfident agent oblivious to the possibility of a hidden vehicle may crash depending on their speed, while an underconfident agent will be overly conservative and proceed with a speed far slower than the minimum necessary to avoid a hypothetical collision.
Intersection (c): The ego vehicle approaches an intersection with a connecting left turn lane, however the lane is occluded by traffic.
An expert agent will slowly proceed through the intersection to avoid a potential collision with an occluded vehicle turning left.
An overconfident agent proceeding will cross the intersection faster, but will be unable to avoid a potential collision with an occluded vehicle attempting to make a turn if one exists.
An underconfident agent will become frozen prior to entering the intersection due to the inability to see oncoming traffic in the connecting lane.
Three-agent intersection (d): In this variation of the intersection, occluded agents may exist to the right and left of the ego vehicle.
The behavior of an expert agent is similar to that in the two-agent variant: the ego must proceed through the intersection slowly enough to emergency brake and avoid collisions with other turning vehicles that are initially occluded by traffic.
4.2 Training and evaluation procedure
To collect observations to train our learned model, we use hand-coded deterministic policies for all agents, which are designed to generate all plausible modes of realistic driver behavior in the scenario (all modes are balanced equally).
At the evaluation stage, we use the same hand-coded deterministic policies for the actors, but use planned trajectory for the ego vehicle with a fixed replanning rate for a fixed horizon T .
Planned waypoints are turned into explicit low-level actions in the CARLA simulator (steering, throttle and brake) using a P-controller (see the supplement for pseudocode describing the control procedure).
We evaluate each method a total of 30 times on each scenario with a balanced split of modes, and report the fraction of episodes where the planner was able to successfully reach the goal (RG) in Tab. 1.
Additionally, we report the fraction of episodes where the planner reached the goal in optimal fashion (RG ∗ ): for example, an overly conservative planner may successfully clear an intersection (RG), but take much longer in doing so than an experienced human driver (RG ∗ ).
Joint trajectories x that satisfy RG ∗ are a strict subset of those that satisfy RG, i.e.,
RG ∗ → RG.
During training and evaluation, we randomize the starting positions of all agents in the scene and randomly drop a fraction of each LiDAR point cloud observation.
The CARLA simulator is an additional source of non-determinism (e.g., slight changes in lighting and foliage).
Single-agent planning baseline: Though it is possible to plan in multi-agent scenarios using a single-agent behavior model, it will often lead to dangerously overconfident behavior, e.g., entering an intersection while disregarding oncoming traffic.
We implement a single-agent (learned) planner baseline using version of our model trained only on ego data (effectively the same model as in [50]).
Multi-agent fully-observed planning baseline:
In partially observed settings, a multi-agent planner that assumes full observability can lead to dangerous overconfident behavior.
For example, the ego vehicle may begin to enter an intersection while oblivious to occluded oncoming traffic; once the ego vehicle begins tracking previously occluded cars, it may be too late to reverse course and avoid a collision.
To implement our multi-agent fully-observed baseline, we train our behavioral model on ground-truth fully observed joint trajectories, and only begin predicting a joint trajectory once a non-ego agent has been detected.
Similar planners can be created by combining any number of modern trajectory prediction methods (that are flexible to varying numbers of agents) with modelbased planning algorithms such as MPC.
Our version of the baseline enables planning contingencies over stochastic behavior, however it does not enable planning with respect to undetected agents.
Multi-agent partially-observed planning baseline: We compare our approach to a recent gametheoretic planner for partially-observed AV settings proposed by Zhang and Fisac [11] (SOAP), which plans with respect to occluded vehicles by formulating the driving scenario as a two-player zero-sum game of pursuit: the ego vehicle computes an area consisting of all points from which another agent could ‘capture’ or collide with the ego (the ‘danger zone’).
The ego vehicle also computes a set of points that a vehicle located just beyond the its field of view could reach in the next t timesteps (the ‘forward hidden set’).
Having computed both these two sets, the ego vehicle can then safely plan by ensuring the danger zone and forward hidden set never intersect.
SOAP generates an expert-level policy similar to AVP, however it requires significant setup per scenario, whereas our method can be easily transferred to new scenarios (assuming expert observations exist).
4.3 Results and discussion
In our experiments, AVP clearly outperforms the single-agent and fully-observed multi-agent baselines, both of which exhibit overconfident behavior leading to collisions with occluded vehicles.
Since our learned model and planner are robust to noise, the planner either clearly succeeds or clearly fails depending on the scenario configuration: in Tab. 1, we can see that the single-agent and multi-agent baselines will consistently fail when another agent is present (due to each scenario’s construction).
On the other hand, both multi-agent partially-observed planners (AVP and SOAP [11]) generate expert-level plans that are contingent on the existence of undetected agents.
Although both AVP and SOAP are capable of generating expert-level plans in two-agent partiallyobserved settings, AVP is a more general solution.
While the game-theoretic baseline has no learned
Two-laneovertake
Blindsummit
Intersection(two-agent)
Intersection(three-agent)
Method
RG ↑ RG* ↑ C ↓ Time ↓ RG ↑ RG* ↑ C ↓ Time ↓
RG ↑ RG* ↑ C ↓ Time ↓
RG ↑ RG* ↑ C ↓ Time ↓ Overlyconservativereferencedriver 30 /0 /0 / 30 26.0s 30 /0 /0 / 30 25.0s 30 /0 /0 / 30 23.5s
30 /0 /0 / 30 28.2s Single-agent(oblivious)[50] 14 /
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
(obliviousuntiltoolate)[6]
[Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
theoretic † (designed,
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
components (and thus does not require an initial training stage), it requires significant setup for each individual scenario: since the planner requires a discrete state space, the reachable areas of the map must first be discretized into a lanelet map.
Additionally, a road centerline for each agent must be specified a priori, since the state of each agent is represented as vertical and lateral positioning relative to a reference trajectory.
Computing the forward hidden set during planning requires a precise measure of the ego agent’s FOV, which is done by placing (observed) dynamic obstacles in a 3D occupancy grid [51], which must be also be generated a priori.
While AVP requires training a behavioral model, assuming a dataset of observations exists or can easily be collected, applying
AVP to new scenarios is straightforward.
AVP predicts and plans continuous positions in Cartesian space, and does not require reference trajectories at test-time based on provided map knowledge.
AVP also does not require a geometric representation of FOV to estimate occlusions, and instead learns to recognize occlusions directly from perceptual context, e.g., LiDAR or camera images.
AVP also scales to scenarios with additional agents: the prediction and planning procedures remain the same as the two-agent variant, and the compute required a single forward pass through the neural network scales linearly with the number of agents.
In contrast, SOAP exhibits overconfident behavior on the three agent scenario, and thus does not plan safe trajectories (as seen in Tab. 1).
Because the
SOAP planner uses a game-theoretic solver that is limited to two agents, scaling the existing SOAP planner to three or more agents can only be done by solving multiple two-player games independently between the ego and non-ego agents.
This independence assumption leads to overconfident driving policies: for example, the ego may exist in a state with a closed-loop escape policy to avoid car 1, but which results in a collision with car 2 (and vice versa).
Limitations:
AVP models detection instead of directly modeling existence, which has the key advantage of enabling training on partially-observed data (further discussed in Sec. 3).
However, a disadvantage of this approach is reduced interpretability:
AVP cannot be probed to visualize the predicted current positions of occluded agents, instead, the model can only express where occluded agents may appear (become disoccluded) in the future.
We anticipate future work will further explore this trade-off between interpretability and tractability for planning under perceptual uncertainty.
Because AVP uses a learned model, it requires a method of collecting training data in the form of partially-labelled agent coordinates (for agents that are detected by the sensor system), and synchronous perceptual context (such as LiDAR or RGB images).
Additionally, the learned trajectory model is object-centric, which (in its current form) does exclude modeling certain phenomena such as objects splitting (e.g., a pedestrian getting out of a vehicle is modeled as a pedestrian appearing out of nowhere).
Both of these limitations are shared by most learned trajectory-modeling techniques.
Conclusion
In this paper we present Active Visual Planning (AVP), a scalable method for contingency planning under multiple forms of partial observability (uncertainty in existence, positions, and intentions) that accommodates unknown dynamics, raw observations, and learning from offline data.
Because AVP is learned purely from observations, it does not require expert action labels or domain knowledge (such as state-space discretization or specification of an approximate dynamics model), making it easily generalizable to new environments and scenarios.
We demonstrate the effectiveness of AVP on a set of autonomous driving environments inspired by real-world scenarios that require reasoning about the existence of other agents, in which AVP achieves expert-level performance.