Generating Scenarios with Diverse Pedestrian Behaviors for Autonomous Vehicle Testing
Authors: Maria Priisalu
Aleksis Pirinen
Ciprian Paduraru
Cristian Sminchisescu
Venue: CoRL2021 Poster (2021)
Keywords: Autonomous Vehicles
AV Testing
Reinforcement Learning

Abstract: There exist several datasets for developing self-driving car methodologies. Manually collected datasets impose inherent limitations on the variability of test cases and it is particularly difficult to acquire challenging scenarios, e.g. ones involving collisions with pedestrians. A way to alleviate this is to consider automatic generation of safety-critical scenarios for autonomous vehicle (AV) testing. Existing approaches for scenario generation use heuristic pedestrian behavior models. We instead propose a framework that can use state-of-the-art pedestrian motion models, which is achieved by reformulating the problem as learning where to place pedestrians such that the induced scenarios are collision prone for a given AV. Our pedestrian initial location model can be used in conjunction with any goal driven pedestrian model which makes it possible to challenge an AV with a wide range of pedestrian behaviors -- this ensures that the AV can avoid collisions with any pedestrian it encounters. We show that it is possible to learn a collision seeking scenario generation model when both the pedestrian and AV are collision avoiding. The initial location model is conditioned on scene semantics and occlusions to ensure semantic and visual plausibility, which increases the realism of generated scenarios. Our model can be used to test any AV model given sufficient constraints.

Generating Scenarios with Diverse Pedestrian Behaviors for Autonomous Vehicle Testing
Maria Priisalu', Aleksis Pirinen", Ciprian Paduraru and Cristian Sminchiseseu 1.5
'Ludd University, 'RISE Research Institutes of Sweden, 'University of Bucharest, *Institute of Mathematics of the Romanian Academy, 'Google Research
Abstract: There exist several datasets for developing self-driving car methodolo- gies. Manually collected datasets impose inherent limitations on the variability of test cases and it is particularly difficult to acquire challenging scenarios, e.g. ones involving collisions with pedestrians. A way alleviate this consider automatic generation of safety-critical scenarios for autonomous vehicle (AV) test- ing. Existing approaches for scenario generation use heuristic pedestrian behavior models. We instead propose framework that can use state-of-ther -orth-f--hheart pedestrian motion models, which is achieved by reformulating the problem as learning where to place pedestrians such that the induced scenarios collision prone for given AV. Our pedestrian initial location model can used in conjunction with any goal driven pedestrian model which makes it possible challenge an AV with a wide range pedestrian behaviors this ensures that the AV can avoid collisions with any pedestrian it encounters. We show that is possible to learn a collision seeking scenario generation model when both the pedestrian and AV are collision avoiding. The initial location model is conditioned on scene semantics and occlusions ensure semantic and visual plausibility, which increases the realism of generated scenarios. Our model can be used to test any AV model given sufficient constraints.
Keywords: Autonomous Vehicles, AV Testing, Reinforcement Learning
Introduction
Research autonomous vehicle (AV) models has gained momentum in recent years [1]. There exist both end-to-end AV models which make decisions directly based on visual sensor outputs [1-6], and hierarchical models which require intermediate processing (such pedestrian detection) of sensor outputs for decision making [7, 8]. To ensure traffic safety, e.g to avoid fatal collisions [9], there is need evaluate the various AV models in criitcy- situations. In this paper we consider safety testing of the full pipeline of perceptive AV models from sensor inputs (e.g. images) to steering. There exist several datasets [10-17] for developing and evaluating AV models, but manually collected data is typically gathered from traffic scenarios that seldom exhibit collision and near-collision scenarios. This shortcoming has lead to recent developments of safety-critical test case generation methods [18-32] for AV models. These existing approaches resort to simulated pedestrians which are not representative of the rich and varied behavior of real pedestrians [33] either the pedestrian trajectories are handcrafted, or the pedestrian models are trained to behave in unnatural ways (e.g. pedestrian agents which are adversarially trained to collide with vehicles). Thus these methods may provide insufficient insights on how the AV would act in scenarios involving real pedestrians. At the same time, there exist a large number of state-of-th of-the-art pedestrian behavior models [34 49] which learn, from real traffic scenarios, how pedestrians interact with the world.
Different from [18-32], we reformulate the problem of generating challenging scenarios as one of learning the distribution y of pedestrian initial locations which are likely to induce collisions between the pedestrian and the AV, for given pedestrian behavior model w. This reformulation allows the use of ofte-of-thhe-n goal driven pedestrian behavior models π in AV test case generation,
Work partially done while at Lund University.
5th Conference on Robot Learning (CoRI. 2021), London, UK.

---

Synthesizer input:
Pedestrian Behavior
Top view of scene
Model π
semantics]
Inital velocitie + positions
Interchangeable Any goal driven
AV, cars and other pedestrians
pedestrian model
Adversarial Test
Goal location (e)
Synthesizer μ
Autonomous Vehicle Model p Interchangesble Any goal driven model Goal location (0)
Synthesizer feedback: Rewards plussibe pedestrian
motion and collisions AV
Synthesizer output: Peddestrian's initial location (eg. MI, N2) + goal (ocation
Figure 1: Overview of the proposed safety-critical test case generation model for AVs. The Adver- sarial Test Synthesizer (ATS) is trained to position a pedestrian with behavior model æ such that the induced scenario likely yield collision with the AV p (car to the right). Four example scenarios are shown. Scenarios #3 and #4 are visually implausible, as pedestrian cannot simply appear from nowhere into the line of sight of an AV. Scenarios #1 and #2 are both plausible and challenging, as the pedestrian is close to the AV and not in the line of sight of the AV due to the occlusions.
which means that the AV can be stress tested in more realistic scenarios compared to prior works. There exist three types of pedestrian behavior models - collision secking, collision ignorant, and collision avoiding each of which gives rise to a distinct optimization problem in our framework. We are the first to show empirically that a non-trite snivtrivial solution exists when the pedestrian model is collision avoiding. Different from previous work [18-32], we explicitly model scene semantics to learn to generate semantically and visually plausible traffic scenarios. Our model can be used to augment existing data by adding simulated safety critical pedestrians to real traffic scenarios.
In real traffic an AV can be expected to encounter pedestrians with a range of different behaviors. Some individuals follow traffic rules and plan their movements based on the surroundings; others are inattentive and take risks. Independently of the pedestrian's 's overall behavior, an AV should able to avoid collisions with the pedestrian when it appears from an occluded space. To ensure this, AV test scenarios should cover the true variation of different pedestrian behaviors. Previous works [18-32] either assume that the pedestrian motion can be modelled by simple constant velocity model, or that the pedestrian motion is adversarial to the AV. In reality however, collisions do not occur only when a pedestrian has perfectly predictable path (e.g. constant velocity), or when the pedestrian is actively seeking to get hit by the AV (e.g. adversarial pedestrian model). Quite the contrary distracted, due to occlusions noise. To alleviate
most collisions occur because pedestrians the previous unrealistic assumptions on pedestrian motion in generative AV testing, we separate the problem of finding the pedestrian location distribution fu from the modelling of the pedestrian behavior. The main problem is then to find a location distribution fu such that the number of collisions between a black-box AV and a black-box pedestrian is maximal in expectation. In AV testing the proposed approach should be used with as many different pedestrian behavior models as possible, as an AV should be seeking avoid collisions with all (even collision seeking) pedestrians.
The pedestrian location distribution 14, shown in Fig. 1, is conditioned on scene semantics, distance to the AV, well dynamic occupancy and occlusion map. Occlusions cause AV to miss a pedestrian (or vise versa) [50] and can thus cause collisions, therefore affecting the shape of μ. Furthermore, / likely to be shaped by the scene semanties (e.g., pedestrians are more likely to reside on sidewalks than on grass) [33, 51, 52]. In previous works, test case generation for AVs has been treated as reinforcement learning problem [20-24, 32] as a black-box optimization problem solved by bayesian optimization (BO) [18, 21]. BO [53] cannot be used to learn la as is inherently discontinuous in realistic scenarios pedestrians can only appear from occluded spaces [51] (cf. Fig. 2). Reinforcement learning (RL) on the other hand does not assume that the policy continuous, and avoids the curse of dimensionality (that occurs classical control and planning methods) in problems, like ours, with large state spaces with unknown world dynamics [54]. We thus propose the Adversarial Test Synthesizer (ATS), an RL agent which positions pedestrians in a given scene (see Fig. 1). It selects initial locations for the pedestrian according to its policy μ., which is optimized to increase the number of collisions. For the ATS agent, the uncontrollable external dynamics include the scene, the AV, and all other pedestrians and cars. We model as heatmap over the scene, parametrized by deep convolutional neural network. Our pedestrian initial location model μ allows collision seeking scenario generation with any goal driven pedestrian behavior model

---

and any AV model. This allows for more varied and more realistic testing of the AV. We show that near-collision scenario generation with a collision avoiding pedestrian gives rise to a previously unstudied optimization problem in AV testing. We show that this problem has a solution.
1.1 Related Work
Previous works [21, 23, 24] have studied the generation of full pedestrian trajectories (zo..... .........yy Tr) for AV testing, such that the trajectory is adversarial to the AV p. This leads the pedestrian only behaving in a suicidal manner. This is unnecessarily limiting as typically it is not only the AV which aims avoid collisions. Ultimately in testing we wish to ensure that the AV can avoid collisions with adversarial as well as collision avoiding pedestrians. In [18] pedestrians are modelled with constant velocity and are initialized from a set of predefined positions. The AV is retrained in a loop with the test case generator. In [21] existing trajectories are adapted to become ad adversarial. The suggested method requires a varied ground truth dataset and the generated data is dependent on the variability of the existing dataset. Similarly, [25, 26, 28-31 augment existing datasets in a latent or trajectory space, which again requires large and varied ground truth dataset.
When only testing the vehicle control of a hierarchical AV system, the set of initial locations that cause collisions our setup since we consider the full stack of the AV, not only the control problem. Moreover, in our framework the pedestrian is not necessarily adversarial to the AV, and the scene dynamics cannot be described by differential game. Our proposed approach allows the testing of AV models with pedestrian models that are semantically aware, collision avoiding, goal reaching and articulated. We do not use robust control methods because we utilize explicit pedestrian behavior models.
be found by the Hamiltonian-Jacoh reachability set [55]. This is not possible in
There are number of recent studies which explore visual relations in data from the AV's perspec- tive [51, 52, 56]. Makansi et al. [51] learn a visual prior for where pedestrians and other objects can appear from the perspective of camera mounted on AV. A similar problem of realistic object placement in LiDAR scenes is studied in [56]. Finally, [52] show that visual cues from an on-board camera can be used to learn walkable areas in a scene. The results of [51, 52, 56] indicate that realistic data contains strong correlations between the scene's 's semantic structure as well as the the presence and behavior of pedestrians and stationary obstacles. We thus include such semantic cues and occlusions in our proposed pedestrian location distribution model μ,, as described in $2.
2 Methodology
In our framework, the ATS and the AV model play an indirect constrained minimax game, and no assumptions are made about the pedestrian behavior model m. Thus и can be cooperative with either the AV ATS jr, or be ignorant with respect to both of these. The problem then becomes a constrained indirect three-agent minimax game with up to two agents per team. The study of the equilibrium [57, 58] is beyond the scope of this paper. However, it is clear that if the AV p. pedestrian behavior N and pedestrian location distribution μ are unconstrained, then the minimax problem has a trivial solution. If the pedestrian is always initialized arbitrarily close to the front of the AV (when the AV's initial velocity is forward), then this will always lead to a collision. If the pedestrian and the AV always stand still or always move in opposite directions, then there are never any collisions. To avoid trivial solutions, sufficient constraints are needed.
To illustrate the minimax problem, assume that the loss functions for the AV the pedestrian and the ATS μ are respectively given by sum of the expectation of the number of collisions and other loss components. Let the number of collisions between given AV and pedestrian be measured by an indicator function 7 that is if a collision occurs and otherwise. The AV model p and the pedestrian location distribution model μ are leart by minimizing the loss functions Jp and Jn respectively,
min Jp = min E [4] + fo(p)) s.t. p E B, P
(1)
min Jn max (E [7] - fx(1)) s.t. p By
(2)
where Jp and Sn are loss components of J, and J,,, respectively. And B, and B, describe the model constraints of p. and j respectively. Equations (1) - (2) express the general optimization problem
when the pedestrian behavior π is independent of B[[] (for example constant velocity R²) If x is
3

---

collision avoiding then (1) (2) together with the following equation describe the general problem
min J - min (Ew.p.=[1)+ + f=(u)) s.t. π E Bry
(3)
x
where f= is the loss component of Jr, and B describes the constraints on the model n. If the pedestrian behavior model π is adversarial then (3) will be replaced by max, (By.p.- + f= (1)) s.l. π E Bz It is clear that the choice of the behavior policy и changes the optimization problem and affects the solutions of μ and p. Depending on the choice of x, the set of applied constraints B3 and Bp may need to be adjusted to ensure that none of the models converge to a trivial solution. Previous works [18-32] have considered the cases where и is adversarial or constant velocity model. In our experiments we illustrate that with sufficient constraints on J4, π and p. a non- con-trivial solution exists for (2) when is collision avoiding. The classical existence conditions of solution of zero-sum game cannot be applied [57, 59] because the problem at hand is not a zero-sum game, as the pedestrian has loss terms f= that are not present in the AV's loss function Jp-
2.1 Special Case: Three Reinforcement Learning Agents
We view the problem of learning the pedestrian initial location distribution reinforcement learning (RL) problem with three agents: the pedestrian, the AV and the ATS. At timestep & E {0,T 1} the pedestrian and the AV move in the scene by taking actions af and of, respectively; we gather these in a joint vector a = (a7 af). The pedestrian's ' action is sampled from the pedestrian policy aj ~ (-I( ) conditioned on its observation sf of the scene which includes the AV. Similarly, the AV chooses actions as of p([lss)) where se is the AV's observation of the scene which includes the pedestrian. The states 8i and sf respectively contain the pedestrian's i and the AV's final goal location. We join oh and sy as vector s; = (sf )²)). The unknown world model p(5t+1 |st, az) provides the transition probabilities from state St to state st41 when the pedestrian and the AV take the joint action az The pedestrian's 's and the AV's actions are evaluated by the reward functions Tx (St, Gt 5t-1) and ry r5(3,, a, 1t+1), respectively. The policies π and p are trained to maximize the respective expected discounted cumulative future rewards (i.e. the utility) at each state St.
We assume that the AV's initial location yo. initial velocity of and final goal location are given. Before the Oth timestep the ATS observes 8 = (S, D, OP). where S is the top view RGB and semantic images of the scene, with constant velocity predicted dynamic occupancy D of the AV(calculated from yo and "f), external cars and external pedestrians in the scene, and OP is the elementwise product between the occlusion map from the AV's perspective O and j prior distribution P. The prior P is heuristic of μu which assigns high probability to pedestrian initial locations that are close to the AV and that can lead to a collision assuming constant motion og of the AV. The ATS agent samples an initial pedestrian location 2o from the policy
), which is the product between OP and the learnable policy μu. To reduce notational clutter we will in $2.1 omit the notation OP from OP and let μ denote the policy of ATS. The pedestrian with an initial location To is given goal location g" and velocity of such that the pedestrian's 's path to g" coincides with the AV's assuming both move with constant velocity. After sampling the pedestrian's initial location 2i simulate the pedestrian at the location 20 with velocity vō- Next we can simulate the pedestrian's 's and the AV's observation of the world sß: S6 t = 0. Our aim is to find the the initial distribution μ i.e. the policy of the ATS agent- - which leads to the highest utility for the reward function (st Qr St+1), where r attains its highest value when the AV and pedestrian collide.
In our experiments the learnable P, n, μ are modelled by policy gradient models and share the loss
J = Exo-ue (oa)).))) sanninin war p.ppsppppppppp. -pseerp(-lsss
Σ >tr( of s141)
(4)
where r = (ru rn.r)) The loss functions' dependence on is expressed in the different re- ward functions. To simplify notations let the state-action history T (an, $1, ar-,, ST), and the discounted cumulative reward R = ETD 3'r(se, at St11). We can express (4) as (r(wo)drddddd" where Pr is the probability density
E[R] = -fa- J2n J, R(zo, ()q(s") (p(())) bd)pp function of given zu, and q is the probability density function of s" Let S be the set of allowed values for (T, xo, 8*)) then for finite T (4) can be rewritten to reveal the relationship between and r,
J= / ((z0o1())
rrr(s, 01 9441)
II k=
x (a]l (aaaaaalass)a ()((s 18k as ddrd (5)
t=

---

Figure 2: Left: A top view image of sample prior P of μ.. In red other pedestrians, and in blue are cars. The prior implies a higher likelihood of pedestrian initial placement which are close to the AV. Right: The same prior after multiplication with the occlusion map O.
Figure 3: Top-view of two different scenarios. The AV is green, external cars and pedestrians are blue and dark red, respectively. In each of the two examples, the left image shows the prior distribution and the right image shows the final initial distribution. Left example: The prior P induces a high likelihood for initializing a pedestrian close the AV, but the probability map is very smeared out, The final distribution Pp. is much less scattered than P and more peaked close to the AV (indicated also with an external orange ellipsoid, more clearly show where the probability mass is). Righr example: We see similar phenomenon as in the left example, in the dense dataset (see supplement).
From the above it is clear that j's step reward is Ry Ei= T (St, at. 8141). We use REINFORCE [60] to find fs. The models / and can be learnt simultaneously as shown in the supplement. If the environment model p is known, we can try find the closed form solution of (5). Using Bellman equations would then allow for a white-box treatment of the AV and the pedestrian.
2.2 Adversarial Test Synthesizer
The ATS is policy gradient agent, with policy μ.. Its objective is to provide an initial position to the pedestrian agent such that the pedestrian collides with the AV. To do so the ATS needs to find locations near the AV where pedestrians and their motion difficult to detect for the AV. To this end the ATS gets an input consisting of a top view image the scene, the prior P of μ that depends on the distance the AV, and the temporal mapping of dynamic objects D. The initial distribution depends also on the scene semantics S, as ATS should learn that pedestrians are more likely to reside near certain semantic classes such as pavement. The policy μ is conditioned on the state s" of size (128 x 256 C)2 where C = 17 is the number of channels. Due to the success of neural networks in vision tasks and the visual nature of the input f is modelled by a two layered convolutional neural network with bi-linear interpolations and a softmax output layer (see Fig. in supplement). The output of the network μ(s²)) is heatmap of size (128 x 226).. The heatmap multiplied by the prior P (see Fig. 2) to avoid sampling zo that cannot possibly lead to a collision. To enforce visual feasibility the ATS can be required sample zo only from locations that are occluded for the AV. This can be done by sampling To OPu(s"") i.e.the product of the occlusion map O, the prior P and μ(s")..
The ATS's reward r evaluates at timestep t the pedestrian's behavior at position Ft. Collisions with all external objects, cars and pedestrians are penalized but collisions with the AV are given a positive reward. Steps di taken in areas often visited by pedestrians rewarded. Steps a, towards the goal gd are rewarded. The reward ri is adapted from Fx $2.3.
2.3 Pedestrian Model
The collision avoiding pedestrian behavior policy is the goal driven Semantic Pedestrian Locomotion model (CARLA SPL) [46]. The a policy gradient agent that is trained by alternatively optimizing for the maximum likelihood objective of pedestrian trajectory forecasting and for the policy gradient objective collision avoidance. The reward function rx (see supplement) of я encourages motion in pedestrian dense areas with the reward term Rped and penalizes collisions with cars (including
5

---

the AV), other pedestrians and static objects in the reward term Reoll. The reward component Rg encourages movement towards the goal location g", and Rg penalizes unnaturally large motions.
The model observes a local crop Se( of size 5m x 5m of the semantic labels and RGB top view image of scene S and local crop (1x))) of the dynamic occupancy map Dz. Further the state St contains history of past actions and poses taken by the pedestrian in the past N = 12 timesteps, the displacement to the closest car di and the displacement to the goal g". In summary Sr = (xx), (r( ai_1... ... o, N; df || - -g" []). The policy gradient model takes a step oj consisting of direction and a speed. The step oī is articulated by the Human Locomotion Network.
Unless otherwise stated the pedestrian models weights are kept constant to not deviate from the learnt pedestrian motion. The CARLA SPL model is trained to avoid collisions with the external cars. The external cars have a lower average speed than the highest possible speed for p. This implies that æ expects p always have the same dynamics its surrounding cars.
2.4 Autonomous Vehicle Model
The AV model p is intentionally simple to illustrate the framework empirically and to avoid making constraining assumptions about the AV. The focus of this work is to show that collision avoiding pedestrian behavior models can be successfully used in autonomous AV test case generation given enough constraints on the problem. The AV is policy gradient model with the state sf = ([] well. de, 5t) at timestep I; where || y/ || is the AV's distance to the pedestrian agent, de is the AV's distance the closest car, and 6, is the AV's intersection with the sidewalk. The AV's speed e is sampled from N (sigmoid(w' (wo s. +6), op), where w, b are learnt weights, and 0p 0.1. The sampled speed Ct is then scaled by the maximal speed of 70km hh. The AV's initial position yo and direction are chosen randomly among the external cars' constant velocity future trajectories.
The AV p is assumed to have constant direction and the policy gradient model controls the speed of the AV. Speed control can be enough to avoid collisions, as the AV can stop or accelerate to avoid a collision. Extending the AV's model to allow directional changes complicates the learning as the AV receives two conflicting objectives: to move to a goal location further ahead and to avoid collisions. The research on AVs deals with balancing such conflicting objectives, and in the future aim to replace the minimal AV model with a state-of-the-art AV model. Replacing the current AV model with a sfate-of-the-ee art AV model requires additional constraints to avoid 0 gradients in early training (as the trained AV model may outperform the untrained ATS).
The reward function rp penalizes the AV for collisions with cars, people and static objects. The AV is penalized for driving on the sidewalk proportionally to the AV's overlap with the sidewalk. To motivate the AV to move, a positive reward is given at the end of the episode for the distance travelled llyo yr |l. The full reward function rp is given in the supplementary.
3 Experiments
We experiment on a dataset gathered from CARLA[61]. Training data is collected from Town 1 and consists of 100 training and 50 validation scenes. The test set consists of 37 scenes from Town 2. For each scene a 3d reconstruction of RGB and semantic segmentation is created from a AV's perspective. In all experiments the scenes 51m x 25.6m are voxelized into 20cm cube voxels. All of the tested ATS models are evaluated and trained with the base AV model. During initial experimentation it was noted that the AV model p had trouble learning collision avoidance without an initializer μ.. The base AV model is trained on two scenes for 200 epochs with a that is trained on the training dataset for 10 epochs. A trajectory length of T 30 is used to train the AV model, and T' 100 is used to train the pedestrian initial distribution models. Each scene is evaluated for 10 episodes with T = 100. During testing the pedestrian and AV models perform the mode and mean actions respectively. The action of the ATS model is sampled. The models are evaluated with three different random seeds and the average and the standard deviation (stdev) of the three runs are reported. The reported metrics are
#. collisions- number of collisions the AV model p has with pedestrians on average. m-entropy entropy of the pedestrian policy during the length of episode.
In Table left the proposed pedestrian initial distribution models OP and Pµu from $2.2 generate more than twice many collisions (std=0.01) sampling zo from the priors P and the occlusion-

---

Table 1: Left: The proposed O and Pμ generate more than twice the collisions compared to the baselines; the heuristics the priors P and OP and the random initialization from occluded spaces Random O. Right: An ablation studying the effect of the prior during the training of μa shows that the is robust to changes in the prior during training as OP and P/u trained with the priors OP and P respectively, and tested with the prior OP, have indistinguishable collision rates (stdev 0.01).
Random O Prior P Prior OP OPp
Pμ
Testing prior OP
Pp
OPp
collisions 0.06
0.10
0.09
0.22
0.24
#. collisions
0.21
0.22
x-entropy 0.85
0.60
0.65
0.25
0.24
T--enropy
0.29
0.25
Table 2: Collision rates of the Pl model trained with collision avoiding. distracted, collision seeking and constant velocity pedestrians. The pedestrian model does not affect the collision rate of the proposed μr, as long as the pedestrian model is not the constant velocity model.
Collision avoiding SPL
Distracted SPL+e
Adversarial SPL A.
Adversarial STPN
Constant velocity HLN
#. collisions T-ennropy
0.21(+0.02)
0.20(+0.01) 0.22(+0003) 0.10(+
0.11(10.022
0.29(+ 0.25(+0.02) 0.029(+0.001) 0.53(#0.03)
masked prior OP. This confirms that z learns and improves beyond the initial prior distribution, and that OPp produces more collisions than the hand-designed heuristies P and OP. The baseline Random O the random initialization of pedestrians from occluded spaces with 360 field of view has the lowest collision frequency. This is likely because occluded spaces may be far from the AV. The proposed OP has a lower r-entropy than the prior OP suggesting that OPμ has learnt to initialize the pedestrian such that the pedestrian's 's direction of movement is as predictable as possible. With low x-entropy has more control over x's trajectory. To the left in Fig. 3 the prior P and the corresponding scene's s Pj distribution are visually compared. The Pu has learnt decisively to initialize the pedestrian near the AV, and with higher probability towards the sidewalk than the road.
In Table left the models OPh ( (.e. μ trained and tested with the prior OP) and Pu (i.e. μ trained and tested with prior P) showed no significant difference. Showing that a 90" occlusion map does not significantly affect μ.. Further applying the occlusion mask O only in testing does not affect the number of collisions, as seen when comparing Pp to OPμ in Table 1 right. This suggests that curriculum learning may be used to enforce larger changes the prior P to facilitate 360 of view occlusion masks (for LiDAR data). A visual comparison of P and OP can be seen in Fig. 2.
In Table 2 the following pedestrian behavior policies are used to train μr, Collision avoiding SPL- - the goal reaching collision avoiding pedestrian model described in $2.3 Distracted SPL+c distracted SPL pedestrian. With a 0.3 probability at each timestep the pedestrian will not notice the AV for m ~ (oisson(2) timesteps.
Adversarial SPL A. an adversarial SPL agent. The SPL model that is finetuned with the R reward. SPL A. is trained simultaneously with (see supplementary Algorithm I with a, = 0). Adversarial STPN A. an adversarial agent that has the Semantic Trajectory Policy Network architecture [46] i.e. the SPL. architecture without the Human Locomotion Network (HLN). The STPN A. is trained from random weights simultancously with μ to maximize the the number or collisions (RsTeN= I from $2). STPN A. is not trained to maximize the negative likeliooddoood of pedestrian trajectories like the SPL models, and it is the only model without locomotion.
Constant velocity CV - constant velocity motion articulated by [62]. The agent moves towards the goal with speed drawn from Gaussian with μ = 1.23ms and = 0.3 [63].
The models Pjs trained with the collision avoiding SPI., the distracted SPL+e, the adversarial finetuned pedestrian policy SPL A. and the adversarial STPN A. (most similar to previous work) are on-par, showing that can learn to control the collision avoiding SPL as well as an adversarial pedestrian model. The collision seeking STPN A. does not outperform the collision avoiding SPL likely due to STPN A. 's high entropy that makes STPN A. hard to control for μ.. STPN A. has no motion priors and can get hit by the AV with motions that have low likelihood in real pedestrian trajectories, such as zigzagging in the middle of the road. Even though the CV model is the most controllable, the initializer trained to control CV results the lowest collision rate because the AV has an easy time avoiding collisions with the CV. The ia trained on SPL+ distracted pedestrian could be expected to have a higher collision rate than SPI., as v has a noisier estimate of the AV's position. Unfortunately μ does not learn to utilize this unnaturally unstructured (and thus unpredictable) noise.
7

---

Figure 4: Sample trajectories of the Simultaneous-p,, model, sub-sampled at 5 frames from frame 0. First row: The AV changes speed thus causing the pedestrian to incorrectly estimate the AV's motion and walk into the AV. Second row: The pedestrian waits for the AV to pass before crossing the road.
This illustrates the need for realistic noise model. Natural noise in the pedestrian's 's observation of the AV could be expected to be more structured, for example high noise levels are expected near occluded spaces. Some sample trajectories of the simultaneously trained μ and p (see supplement) shown in Fig. a collision prone initialization, and an initialization that does not lead to a collision because the AV speeds away.
4 Conclusions and Future Work
We are the first to utilize state-of-the-ar pedestrian forecasting models in generative AV testing. We have proposed a general framework that capable of stress testing the collision avoidance of AVs with wide range of pedestrian behavior models. In practice we wish to ensure that an AV can avoid collisions with all pedestrians (intoxicated, law-obedient, children etc.), and thus should test the AV with as many different pedestrian behaviors possible. Our empirical evaluations show that goal driven pedestrian model with any behavior can used in this framework. This is significant result, as no prior work has shown that a collision avoiding pedestrian model can be used to generate collisions with a collision avoiding AV. To achieve this, we have proposed the Adversarial Test Synthesizer (ATS) which, given any goal driven pedestrian model, learns the pedestrian initial location distribution μ that maximizes the expected number of collisions with a given AV. The ATS is modelled by neural network which receives input the top view image of the scene, the scene semantics, the occupancy of dynamic objects, and outputs a distribution ja over pedestrian initial locations. We have shown that f can learn to adversarially position collision avoiding pedestrian model that has been trained on ground truth pedestrian data and obeys human locomotive dynamics. Our work, for the first time, shows that generative models of AV test scenarios can utilize state-of-the-art pedestrian motion models instead of the typically used models which do not resemble real pedestrian motion. Stress testing AVs with state-of-the- ofrthe--rt pedestrian forecasting models decreases the statistical difference between tested and real pedestrian behaviors, which could reduce the likelihood of real life AV crashes.
We have shown that a learnable pedestrian initial location distribution μ exists for stress testing a basic AV model. Ultimately we wish to extend the result to sate-of-the-an AV models. Since the model ji treats the AV and the pedestrian agent as black- boxes, μ can be trained to adjust to the dynamics of a more sophisticated AV is. However, finding a non-trivial solution will require careful readjustment of the choice of sufficient but realistic constraints. The problem can be constrained spatially by tight streets, occlusion dense scenes, lack of space due traffic density, or by setting a limit on the pedestrian's 's maximal distance to the AV. Alternatively, the pedestrian model can be constrained by adjusting the noise level of the pedestrian's 's internal prediction of the AV's future motion, the noise level of the pedestrian's 's observation of the AV, or the pedestrian's 's goal, dynamics or personality. Similarly, the noise the AV's observation of pedestrians and other traffic participants and their motion should be high enough to lead to collisions. As seen in the experiments, unstructured noise cannot be utilized by μ. thus careful modelling of the noise of the chosen AV's 's observations required. In future work the pedestrian's 's internal prediction of the AV's motion could be impaired with psychologically or physiologically inspired noise process.
8

---

Acknowledgments
This work was supported by the European Research Council Consolidator grant SEED, CNCS- UEFISCDI PCCF-2016-0180, and the Swedish Foundation for Strategic Research (SSF) Smart Systems Program. We would also like to thank the anonymous reviewers for their useful comments.
References
[1] Z. Zhu and H. Zhao. A survey of deep rl and il for autonomous driving policy learning, arXiv preprint arXiv:2101.0199 101101993, 2021.
[2] F. Ye, S. Zhang, P. Wang, and C.-Y. Chan. A survey of deep reinforcement learning algorithms for motion planning and control of autonomous vehicles. arXiv preprint arXiv:210S. 14218, accepted in IEEE Transactions on intelligent transportation systems., 2021.
[3] A. Sadat, M. Ren, A. Pokrovsky, Y--.. Lin, E. Yumer, and R. Urtasun. Jointly learnable behavior and trajectory planning for self-driving vehicles. 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3949-3956. IEEE, 2019.
[4] W. Zeng, S. Wang, R. Liao, Y. Chen, B. Yang, and R. Urtasun. Dsdnet: Deep structured self-driving network. In Proceedings of European Conference on Computer Vision (ECCV).
pages 156-172. Springer, 2020.
[5] D. Chen, B. Zhou, V. Koltun, and P. Krähenbühl. Learning by cheating. In Proceedings of Conference on Robot Learning (CoRL), pages 66-75. PMLR, 2020.
[6] M. Toromanoff, E. Wirbel, and F. Moutarde. End-to-end model-free reinforcement learning for urban driving using implicit affordances. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7153-7162, 2020.
[7] B. Paden, M. Čáp. S. Z. Yong, D. Yershov, and Frazzoli. A survey of motion planning and control techniques for self-driving urban vehicles. IEEE Transactions on intelligent vehicles, 1 (1):33-55, 2016.
[8] Q. Dai, X. Xu, W. Guo, S. Huang, and D. Filev. Towards systematic computational framework for modeling multi-agent decision-making at micro level for smart vehicles in a smart world. Robotics and Autonomous Systems, 144: 103859, 2021.
[9] P. Kohli and A. Chadha. Enabling pedestrian safety using computer vision techniques: A case study of the 2018 uber inc self-driving car crash. In Proceedings of Future of Information and Communication Conference (FICC) 2019, Lecture Notes in Networks and Systems, vol 69.
[10] M.-F Chang, J. Lambert, P. Sangkloy, J. Singh, S. Bak, A. Hartnett, D. Wang, P. Carr, S. Lucey, D. Ramanan, al. Argoverse: 3d tracking and forecasting with rich maps. In Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[11] A. Geiger, P. Lenz, C. Stiller, and R. Urtasun. Vision meets robotics: The KITTI dataset. I. J. Robotics Res., 32(1111121111237, 2013.
[12] P. Sun, H. Kretzschmar, X. Dotiwalla, A. Chouard, V. Patnaik, P. Tsui, J. Guo, Y. Zhou, Y. Chai, B. Caine, V. Vasudevan, W. Han, J. Ngiam, H. Zhao, A. Timofeev, S. Ettinger, M. Krivokon, A. Gao, A. Joshi, Y. Zhang, J. Shlens, Z. Chen, and D. Anguelov. Scalability in perception for autonomous driving: Waymo open dataset, 2019.
[13] B. Zhou, H. Zhao, X. Puig, T. Xiao, S. Fidler, A. Barriuso, and A. Torralba. Semantic understanding of scenes through the ade20k dataset. International Journal of Computer Vision (IJC)), 2018
[14] H. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, and O. Beijbom. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[15] J. Behley, M. Garbade, A. Milioto, J. Quenzel, S. Behnke, C. Stachniss, and J. Gall. Se- mantickitti: A dataset for semantic scene understanding of lidar sequences. In Proceedings of the IEEF International Conference on Computer Vision (ICCV), 2019.
[16) X. Huang, X. Cheng, Q Geng, B. Cao, D. Zhou, P. Wang, Y. Lin, and R. Yang. The apolloscape dataset for autonomous driving. Proceedings of IEEF/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2018

---

[17] R. Kesten, M. Usman, J. Houston, T. Pandya, K. Nadhamuni. A. Ferreira, M. Yuan, B. Low, A. Jain, P. Ondruska, al. Lyft level av dataset 2019. level5. lyft. com/dataser, 2:5, 2019.
[18] Y. Abeysirigoonawwrdenaa F. Shkurti, and G. Dudek. Generating adversarial driving scenarios in high- fidelity simulators. In Proceedings of International Conference on Robotics and Automation, ICRA 2019, Montreal, QC, Canada, May 20-24, 2019, pages 8271-82717 IEEE, 2019.
[19] A. Hamdi, M. Mueller, and B. Ghanem. SADA: semantic adversarial diagnostic attacks for autonomous applications. In Proceedings of Thirty-Fourr Fourthh Conference on Artificial Intelligence (AAAI), pages 10901-10908. AAAI Press, 2020.
[20] P. Gupta, D. Coleman, and J. Siegel. Towards safer self-driving through great pain (physically 2020.
adversarial intelligent networks). ArXiv,
[21] Wang, A. Pun, J. Tu, S. Manivasagam, A. Sadat, S. Casas, M. Ren, and R. Urtasun. Ad- vsim: Generating safety-critical scenarios for self-driving vehicles. Proceedings of IEEF/CVF Conference on Compuler Vision and Pattern Recognition (CVPR), 2021.
[22] D. Karunakaran, S. Worrall, and E. Nebot. Efficient falsification approach for autonomous vehicle validation using a parameter optimisation technique based on reinforcement learning, 2020.
[23] D. Karunakaran, S. Worrall, and E. M. Nebot. Efficient statistical validation with edge cases to evaluate highly automated vehicles. In 23rd IEEE International Conference Intelligent Transportation Systems, ITSC 2020, Rhodes, Greece, September 20-23, 2020, pages 1-8. IEEE, 2020.
[24] W. Ding, B. Chen, M. Xu, and D. Zhao. Learning to collide: An adaptive safety-critical scenarios generating method. In IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2020, Las Vegas, NV, USA, October 24, 2020 January 24, 2021, pages
2243-2250 IEFE, 2020.
[25] W. Ding, B. Chen, B. Li. J. Eun, and D. Zhao. Multimodal safety-critical critical scenarios generation for decision decision-making making algorithms evaluation. IEEE Robotics and Automation Letters, 6(2):1551- 1558, 2021.
[26] P. Parashar, A. Cosgun, A. Nakhaei, and K. Fujimura. Modeling preemptive behaviors for uncommon hazardous situations from demonstrations. CoRR, ttss/800.001. 2018.
[27] M. Wen, J. Park, and K. Cho. A scenario generation pipeline for autonomous vehicle simulators. Human-centric Computing and Information Sciences, 10, 12 2020.
[28] A. Demetriou, H. Alfsvåg, S. Rahrovani, and M. Chehreghani. A deep learning framework for generation and analysis of driving scenario trajectories. ArXiv, abs/2007. 14524, 2020.
[29] D. Nishiyama, M. Y. Castro, S. Maruyama, S. Shiroshita, K. Hamzaoui, Y. Ouyang. G. Rosman, J. A. DeCastro, K. Lee, and A. Gaidon. Discovering avoidable planner failures of autonomous vehicles using counterfactual analysis in behaviorally diverse simulation. In 23rd IEEE In- ternational Conference on Intelligent Transportation Systems, ITSC 2020, Rhodes, Greece, September 20-23, 2020, pages 1-8. IEFF, 2020.
[30] W. Ding, M. Xu, and D. Zhao. CMTS: A conditional multiple trajectory synthesizer for generating safety-critical driving scenarios. In 2020 IEEE International Conference on Robotics and Automation, ICRA 2020, Paris, France, May 31 August 31, 2020, pages 4314-4321, IEEE, 2020.
[31] Z. W. Sun X.. Zhang Y. Building narrative scenarios for human- autonomous vehicle interaction research in simulators. Advances in Simulation and Digital Human Modeling. 1206, 2020.
[32] M. Koren and M. J. Kochenderfer. Efficient autonomy validation in simulation with adap- tive stress testing. In 2019 IEEE Intelligent Transportation Systems Conference, ITSC 2019, Auckland, New Zealand, October 27-30, 2019, pages 4178-4183. IEEE, 2019.
[33] A. Rasouli and J. K. Tsotsos. Autonomous vehicles that interact with pedestrians: survey of theory and practice. IEEE transactions on intelligent transportation systems, 21(3):900-918, 2019.
[34] A. Rudenko, L. Palmieri, M. Herman, K. M. Kitani, D. M. Gavrila, and K. O. Arras. Human motion trajectory prediction: survey. The International Journal of Robotics Research, 39()): 895-935, 2020.
10

---

[35] P. Dendorfer, A. Osep, and L. Leal-Taixe Goal-gan: Multimodal trajectory prediction based on goal position estimation. In Proceedings of the Asian Conference on Computer Vision (ACCV). November 2020.
[36] A. Gupta, J. Johnson, L. Fei-Fei, S. Savarese, and A. Alahi. Social gan: Socially acceptable trajectories with generative adversarial networks. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
[37] V. Kosaraju, A. Sadeghian, R. Martín-Martín, I. Reid, H. Rezatofighi, and S. Savarese. Social- bigat: Multimodal trajectory forecasting using bicycle-gan and graph attention networks. In NeurIPS, 2019.
[38] L. Zhang, Q. She, and P. Guo. Stochastic trajectory prediction with social graph network. CoRR, abs/1907. 10233, 2019.
[39] Y. Huang. H. Bi, Z. Li, T. Mao, and Z. Wang. Stgat: Modeling spatial-temporal interactions for human trajectory prediction. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), October 2019.
[40] A. Mohamed, K. Qian, M. Elhoseiny, and C. Claudel. Social-stgenn: A social spatio-temporal graph convolutional neural network for human trajectory prediction. In Proceedings of IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), June 2020.
[41] A. Alahi, K. Goel, V. Ramanathan, A. Robicquet, F. Li, and S. Savarese. Social LSTM: human trajectory prediction in crowded spaces. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[42] N. Lee, W. Choi, P. Vernaza, C. B. Choy, P. H. Torr, and M. Chandraker. Desire: Distant future prediction in dynamic scenes with interacting agents. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[43] W. Luo, B. Yang, and R. Urtasun. Fast and furious: Real time end-to-end 3d detection, tracking and motion forecasting with a single convolutional net. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018
[44] T. Zhao, Y. Xu, M. Monfort, W. Choi, C. Baker, Y. Zhao, Y. Wang, and Y. N. Wu. Multi-agent tensor fusion for contextual trajectory prediction. In Proceedings of IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2019.
[45] A. Sadeghian, V. Kosaraju, A. Sadeghian. N. Hirose, H. Rezatofighi, and Savarese. Sophie: An attentive gan for predicting paths compliant to social and physical constraints. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[46] M. Priisalu, C. Paduraru, A. Pirinen, and C. Sminchisescu. Semantic synthesis of pedestrian locomotion. In Proceedings of the Asian Conference on Computer Vision (ACCV), November 2020.
[47] Kothari, S. Kreiss, and Alahi. Human trajectory forecasting in crowds: A deep learning perspective. JEEE Transactions on Intelligent Transportation Systems, 2021.
[48] C. Yu, X. Ma, J. Ren, H. Zhao, and S. Yi. Spatio-temporal graph transformer networks for pedestrian trajectory prediction. In Proceedings of European Conference on Computer Vision (ECCV), pages 507-523. Springer, 2020.
[49] Y. Liu, Q. Yan, and A. Alahi. Social nce: Contrastive learning of socially-aware motion representations. arXiv preprint arxiv:20/2. 2020.
[50] X. Ren, T. Yang, L. E. Li, A. Alahi, and Q. Chen. Safety-aware motion prediction with unseen vehicles for autonomous driving. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2021.
[51] O. Makansi, Ö. Çięek, K. Buchicchio, and T. Brox. Multimodal future localization and emergence prediction for objects in egocentric view with a reachability prior. In Proceedings of IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 4353-4362. IEEE, 2020.
[52] Sun, H. Averbuch-Elor, Q. Wang, and N. Snavely. Hidden footprints: Learning contextual walkability from 3d human trails. In A. Vedaldi, H. Bischof, T. Brox, and J. Frahm, editors, In Proceedings of European Conference on Computer Vision (ECCV), volume 12363 of Lecture Notes in Computer Science, pages 192-207. Springer, 2020.
11

---

[53] J. Mockuu. On bayes methods for seeking an extremum. Avtomatika i Vychislitelnaja Technika, 3:53-62, 1972.
[54] D. Bertsekas. Reinforcement and Optimal Control. Athena Scientific, 2019.
[55] S. Bansal, M. Chen, S. Herbert, and C. J. Tomlin. Hamilton-jacbbi reachability: A brief overview and recent advances. In Proceedings of the IEEE 56th Anmual Conference on Decision and Control (CDC), pages 2242-22222 2242-2253, IEEE, 2017.
[56] S. Manivasagam, S. Wang, K. Wong, W. Zeng, M. Sazanovich, S. Tan, B. Yang, W. Ma, and R. Urtasun. Lidarsim: Realistic lidar simulation by leveraging the real world. In Proceedings of IEEE/CVF Conference Computer Vision and Puttern Recognition (CVPR), pages 11 11164 11173. IEEE, 2020).
[57] K. Zhang, Z. Yang, and 'T. Başar. Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of Reinforcement Learning and Control, pages 321-384 2021.
[58] Q. Dai, X. Xu, W. Guo, S. Huang, and D. Filev. Towards systematic computational framework for modeling multi-agent decision-making at micro level for smart vehicles in a smart world. Robotics and Autonomous Systems, 144: 103-859, 2021. ISSN 0921-8890.
[59] E. Smolyakov. Necessary and sufficient conditions for the existence of saddle point in parametrized programmed differential games. Cybernetics and Systems Analysis, 33(5):686
693, 1997. ISSN 10600396.
[60] R. J. Williams. Simple statistical trudient-followiin algorithms for connectionist reinforcement learning. Machine learning, 1992.
[61] A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun. Carla: An open urban driving simulator. In Proceedings of Conference on Robotic Learning (CoRL), 2017.
[62] D. Holden, T. Komura, and J. Saito. Phase-functioned neural networks for character control. ACM Trans. Graph. 36(4):42:1-42:
2017.
[63] S. Chandra and A. K. Bharti. Speed distribution curves for pedestrians during walking and crossing. Procedia-Sone and Behavioral Sciences, 104:660-661) (676606667660 2013.
12


Reviews:
[{"review_id": "bjFRlDE1aQe", "reviewer": "Reviewer_7Zap", "summary": "The paper proposes a learning-based model to generate adversarial pedestrian generation for autonomous vehicle testing. The problem is formulated as learning a pedestrian placement model such that the expected number of collisions between the pedestrian and the vehicle is maximized. REINFORCE algorithm is used to train the model by considering the pedestrian and the vehicle behavioral policies as unknown dynamics, i.e. treating the pedestrian and the vehicle as black-box models. The model learns the pedestrian initial spatial distribution based on the scene semantic, distance to the vehicle, occlusion space, and occupancy map.  The model is trained using simulated data collected from CARLA. The results show that the proposed model can result in a higher number of collisions compared to random initialization and other baselines. The sensitivity of the pedestrian placement model to the pedestrian behavior model is also studied by considering several different behavioral policies. The result shows low sensitivity in most cases.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "As the main strengths, the paper targets the problem of autonomous vehicle evaluation, which is an important and less studied subject in autonomous driving research. The introduction provides a convincing motivation behind the work, and the paper is generally well-written, detailed, and convincing. The supplementary materials also provide further details about the model architecture, input/output, rewards, and the algorithm implementaion. The paper seems to be an initial work that can potentially encourage further research on this subject. \nThe main weakness is that the approach is still limited to a single pedestrian placement as opposed to considering a variable number of pedestrians or a flow of pedestrians. The placement model is interesting, but a true test may require a combination of both pedestrian initialization and behavioral model. Since there is still a dependency on the pedestrian and vehicle behavioral policies, there is a concern that the placement model becomes dependent on those policies, contrary to being a part of an independent evaluation/test system for autonomous vehicels. Lastly, the abstract and conclusion do not seem to converge to a single problem. Abstract mentions generating maximally challenging scenarios for autonomous vehicle testing as the main problem. The paper however concludes that the pedestrian initial distribution and behavior can be learned, but how to evaluate an autonomous vehicle is still not addressed in the paper. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper is an initial work on adversarial pedestrian generation for autonomous vehicle testing. Certainly, not everything can be considered in a single paper and there is room for improvement. The main concern is that the paper is too focused on a single pedestrian initiation/behavior problem, which could also be addressed through some other techniques (e.g. model predictive scheme or some heuristic approaches) as opposed to considering a more comprehensive test generation concerning pedestrian-vehicle interactions.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "Very good: Comprehensive knowledge of the area"}, "extra_text": {"issues": "- The idea of generating challenging scenarios is interesting, but the challenging scenario seems to be narrowed down to adversarial trajectory crossing scenarios or maximizing the number of collisions. This might be a limiting assumption as it does not consider other aspects of autonomous vehicle testing. For example, a vehicle might be able to avoid a collision at the cost of sharp, sudden maneuvers. Or a vehicle might go off-road or collide with other vehicles in reaction to a pedestrian motion. In short, how to define a challenging scenario and how to evaluate an autonomous vehicle based on the test result?\n- The problem formulation is convincing but the use case is still not completely clear. If the goal is to generate test cases, the result may need to be compared with manually generated test cases either heuristically or by filtering randomly generated scenarios. If the goal is to generate collision-prone smart agents then how it is better than a model predictive system that is designed to collide with another agent. Since the dynamics of a vehicle can be approximated or assumed to be known, a simple model predictive scheme might also be able to generate good initial/behavior distribution. \n- The distance unit is not clear in Table 1 and 2. Is it in centimeter or meter?\n- It's recommended to add a list of notations and abbreviations as there are too many abbreviations in the paper.\n- Line 218, type: mdoel >> model\n"}}, {"review_id": "VqhfGnqawVq", "reviewer": "Reviewer_s5dQ", "summary": "This paper proposes a reinforcement-learning based approach for learning a distribution over initial pedestrian locations which are challenging for a nearby autonomous vehicle. The core idea is to separate learning the distribution of initial pedestrian locations that lead to collisions from the human behavior model. This approach is compared in the CARLA simulatior with alternative methods such as randomly generating pedestrian locations or generating them in occluded spaces.  \n", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The problem of interest in this paper -- generating challenging simulated scenarios for autonomous vehicle testing -- is critically important and underexplored\n- The idea of characterizing a distribution over initial pedestrian states which would induce a challenging scenario for the AV is interesting\n\nWeaknesses:\n- The proposed approach of decoupling the unsafe initial state distribution from the human behavior model requires more justification. Furthermore, there are too many confounding variables in the design of the proposed approach which make the utility of the overall pipeline difficult to evaluate. \n- It is unclear how realistic the proposed approach is for modern autonomous vehicles. More details provided below in the detailed comments section. \n- The clarity of the writing (both from a technical and also narrative perspective) could be improved. \n\nDetailed comments:\n \nSection 1:\n- Figure 1. Why does the AV input not include the history of the agent’s movement/poses (if they are not occluded)? This seems key since a core ability of the AV is re-planning and updating its predictions conditioned on the history of the human. \n- Figure 1 caption states that Scenarios #3 and #4 are illegal, but that seems like a misnomer. They aren’t illegal, but rather infeasible (under the laws of physics) for a pedestrian to “simply appear from nowhere into the line of sight of an AV”.\n- “We have strong reason to believe that the optimal pedestrian distribution \\mu is not easy to model…” This sentiment would be strengthened with a more substantive, quantitative definition of “easy to model”. For example, does this refer to the difficulty in choosing the correct features which influence the pedestrian distribution? Does this refer to the dimensionality of the distribution? \n- “... the pedestrian location distribution \\mu is a learnable stochastic policy” The term policy canonically refers to a mapping from states to a (distribution over) actions. With this definition, it is unclear how a distribution over initial states is a policy. Clarifying this use of language would improve this sentence. \n-  “This allows us to model \\mu as a convolutional neural network \\mu_\\Theta” The use of a neural network to model the initial distribution is not well motivated by the preceding paragraphs. For example, alternative methods for computing initial conditions from which safety-critical scenarios can happen include robust-control methods like Hamilton-Jacobi reachability, which compute challenging initial states via the solution to a dynamic programming problem. In the stochastic realm of approaches, the distribution over initial states could be parameterized by a mixture of Gaussians instead of a neural network, perhaps even enabling more sample efficiency. Discussing why a neural network model is essential for representing \\mu would strengthen the motivation and technical approach of this work. \n- In the introduction, works which model adversarial (i.e., “suicidal”) pedestrians are referenced and this work seeks to differentiate itself from these works. However, in the second to last paragraph of Section 1, the pedestrian behavior “can be adversarial to the car or the pedestal location distribution \\mu, or even not directly cooperative to either side.” This appears to be modelling an adversarial setting, which is precisely what the introduction was claiming was an undesirable modelling paradigm. Clarifying this is essential for understanding the difference between the proposed method’s philosophy and existing work. \n\nSection 2:\n- “When the objective J is maximized…. Then both pi and mu become adversarial to the car’s objective J_\\rho”. Notationally this sentence could be made more consistent: the car’s objective is referred to as J^p (not J_\\rho) throughout the rest of the text. The reason behind why this statement is true is unclear due to the fact that J is not described, the relation between J and J_\\rho is not established, and it is unclear where the adversarial nature of pi and mu come into play. \n- “This leads to the pedestrian only behaving suicidal.” Grammatical reworking of this sentence: “This leads to the pedestrian behaving in a suicidal manner.”\n- “To allow for both, we separate nu explicitly into the initial pedestrian locations … and the pedestrian behavior pi.” This is one of the core ideas behind this paper. However, it is not clear why this decoupling should work. More specifically, the same initial pedestrian locations could (a) cause collisions with the AV or (b) not cause collisions with the AV under different pedestrian behavior models and different autonomous vehicle planning/control architectures. For example, a human standing in front of a car is doomed for collision if the human moves towards the car and the car continues forward. But, a human standing in front of a car isn’t doomed for collision if the car reverses and the human runs away. This implies that the distribution over “dangerous” initial pedestrian locations is conditioned on the human policy as well as the autonomous vehicle policy, i.e., mu(S, O, D, P, pi_human(s, g), pi_AV(s)) where pi_human is the human’s policy and pi_AV is the autonomous vehicle’s planner/policy. The justification behind why decoupling the pedestrian behavior and the dangerous initial locations is a reasonable modelling assumption should be improved to increase the overall quality / clarity of the technical contribution. \n\nSection 3:\n- “...pi is constant or independent of E[I].” This sentence is missing a period at the end. \n- “If the pedestrian behavior model \\pi … then the third equation will maximize…” There are only two equations stated in Section 3, so it is unclear what third equation this sentence is referring to. \n- p(s^\\rho_t) is the goal-driven car model, but it does not appear to be goal-dependent like the pedestrian model. Since an initial pedestrian state’s safety depends not only on where the pedestrian is trying to move but also where the AV is trying to move, it appears critical that the car’s model also captures how the AV behaves differently as a function of it’s own goals. Furthermore, it appears that the car model only depends on the state of the ego vehicle, s^p_t. Thus, it is not possible for the autonomous vehicle to react to / predict the pedestrian since it never has access to the pedestrian’s state, which is an unrealistic modelling assumption of an AV autonomy stack. \n- The external world dynamics are denoted by p(s_t+1 | s_t) but this appears to be an autonomous system without any input. Since the text states that these dynamics also include the AV and the pedestrian and any other traffic participants, then it must also include the actions of all other agents too: i.e., p(s_t+1 | s_t, a_pedestrian, a_AV, a_other_traffic_participants).\n- There is a notational inconsistency in equation (3) and (4) when referring to a^p_t and a^c_t -- I believe these should be a^pi and a^\\rho respectively.  \n- In equation (4) is the policy pi(a^p_t | s_t) the same pedestrian behavior model pi(s^pi, g^pi)? The notational inconsistency makes it unclear to understand what this part of the equation represents. A similar question can be asked of the quantity p(a^c_t | s_t). \n\nSection 3.3:\n- “The car model is intentionally simple… The car is a policy gradient model…” Since the safety of a pedestrian’s initial condition and the car’s policy are inextricably intertwined, the safety assessment of the initial conditions only holds under an appropriate choice of the car model. Realistic autonomous vehicle stacks are incredibly complicated. To assess the impact / utility of the proposed framework, a more realistic car model should be discussed or, ideally, evaluated.\n\nSection 3.4: \n- “It should be noted that the CARLA … is trained to avoid collisions on a dataset where cars have much lower average speed…. This provides sufficient constraints on the collision avoidance abilities of \\pi..” This claim is not well justified. Since there is a model mismatch between the simulated human’s policy with respect to other cars, then the human policy may not appear as collision-avoiding as claimed. For example, if the human policy only had to avoid slow-moving cars, then the human policy may not run out of the way of an on-coming faster vehicle, the way a truly cooperative collision-avoiding agent would. This is a confounding variable in the setup of the simulator, and should be either removed as a confound (by using an appropriate pedestrian policy that can truly react to the car in a cooperative way) or it’s effects in the construction of distribution over unsafe initial states should be further discussed / controlled for.\n\nSection 4:\n- There is a typo in Table 1 -- OP should be PO. \n- The text references a distribution P\\mu-D but this is not described in Table 1. Perhaps this is a typo on the far-right distribution P-D?\n- “... m ~ Po(2)” does Po refer to probability distribution? And what does the argument of 2 mean? Please clarify. \nSection 5:\n- Typo “mdoel” should be “model” and “coo-operative” should be “cooperative”.  \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The problem of interest in this paper is very important. However, as currently written, the proposed approach and results are too preliminary and difficult to assess. This paper would be better contextualized as a Blue Sky paper focused on (1) clearly outlining what it takes to do “good” simulation / stress testing of safety-critical scenarios and (2) reworking the proposed approach as a first step towards the simulated, automated stress testing of AV safety systems. Clearly describing (1) is something that is still lacking in the literature, and could be a contribution on its own; after this, concrete instantiations of AV safety testing frameworks -- like perhaps the one proposed -- can be better contextualized. \n\n================\nAfter the discussion period, I have decided to increase my score to \"weak accept\". ", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Fair: Major revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "Very good: Comprehensive knowledge of the area"}, "extra_text": {"issues": "- Please provide further justification for the proposed idea of decoupling the initial states from the human behavior policy (and the autonomous vehicle model). See the detailed comments above. \n- Please clarify the writing and notational consistency so the proposed method can be more clearly understood. "}}, {"review_id": "2ASJHs7dJk", "reviewer": "Reviewer_UAuu", "summary": "This paper considers the online/offline construction of a dataset of challenging scenarios that could support/enhance the training of collision avoidance policies for autonomous vehicles in urban scenarios involving realistic pedestrians. In contrast to existing approaches that either construct such training scenarios out of simplified (constant-velocity motion model) or \"suicidal\" (collision-seeking) pedestrians, this work proposed a novel learning based method to initialize pedestrians and to train a richer set of pedestrian motion models, while still keeping the level of collision (i.e., the overall difficulty of the challenge for the AVs) high. Results are shown in a variety of realistic urban scenes, using different initialization methods and pedestrian policies. There, authors even show that, under certain assumptions, collision-avoiding pedestrians can still lead to the most collisions with AVs when initialized at the right places in a carefully arranged world.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "First of all, this paper is well written, and pleasant to read. I did notice a few typos or grammatical mistakes here and there (see list below), but nothing major. However, it took me multiple pages until I finally started to understand the actual premise of the paper, and I believe this is the main shortcoming of this work for now, but which can be addressed.\n\nIn my opinion, the need for non-suicidal, yet non-constant-velocity pedestrian models is currently weak in the first half of the paper. After my initial read of the abstract and introduction, I believe that I agreed with this premise (on which the entire paper relies), but would not have been able to articulate why or provide any form of example of such a scenario (i.e., non-trivial, non-suicidal pedestrian having a higher likelihood of collision with an AV).\nThen, at the end of Section 3 (line 108-109), I found this sentence that finally mentions that an example of such a non-trivial scenario will be provided later in the experiments section. However, I believe this point really needs to be made clear much earlier in the text. In particular, as a reader, I believe that I would have benefited from a lot more intuition as to what class of examples of this type exist, as the paper heavily relies on this premise. For now, the paper reads a little like \"Please believe us that this very interesting, non-trivial, yet counterintuitive statement is true; we will describe a method to fix it, and then at the full end provide you with one line of a table that shows that we were right (and a few sentences in the text),\" and I worry this structure might not be best in supporting this premise that carries the entire paper.\nAgain, I can believe authors are indeed correct with their premise, which would make the work valuable and nuanced and interesting, but as a reader I would have loved to be taken in through this journey, rather than remain a spectator/believer until the \"big reveal\" (which also needs to be a much bigger/stronger reveal, if this is the main meat of the paper). I believe this is mostly a writing exercise, but one that will be crucial in bringing this paper to the necessary level to be understood/enjoyed by the community and be ready for publication. That is my one major concern with this paper in its current form.\n\n\nOther than that, I only have a few minor comments, as the rest of the paper is very clear and articulates the method and all the assumptions made very clearly and convincingly:\n\n1. Table 2: many of these values seem to be extremely close to each other, and I worry that these results are not statistically significant. Can authors present any form of analysis, or at least average error/standard deviations to help the reader form a more complete opinion of these results? These additional information should likely be discussed in the text as well.\n\n2. List of typos/grammatical mistakes I caught (please perform a full spell/grammar check anyway, I am not claiming to have caught all of them of course):\n\t- line 154: \"the \\mu\" --> \\mu\n\t- line 154: 3m^{-1} --> should this rather be 3ms^{-1}? 3 1/m does not look like a velocity.\n\t- line 181: \"encure\" --> ensure?\n\t- line 221: \"to fraction\" --> to the fraction\n\t- line 243: could be instead be --> could instead be\n\t- line 253: the scenes --> the scenes'\n\t- line 318: mdoel --> model\n\t- line 321: coo-operative --> cooperative\n\n3. The conclusion currently is very short and only summative, and therefore reads as extremely weak. It provides little over a redundant summary of the work, and no intuition or opening to future works, deeper reflection, etc. I worry it was put together rather hastily (as seen from the big typos in it), and believe it could be rewritten to better use this space to conclude the paper (key insight/take home message, deeper reflection, future works).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "In summary, this work proposes a new method to create challenging pedestrian-vehicle scenarios for AV training using more realistic/nuanced, learned pedestrian motion models. The main premise of the paper is interesting and thought-provoking, but really needs to be expanded upon (provide intuition/example) earlier in the text, and again highlighted more thoroughly in the results section. Provided that this is addressed, I am in favor of accepting this work for presentation at CoRL 2021.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Fair: Major revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "issues": "Issues were listed above in my main review (major and minor ones).", "reviewer_expertise": "Good: General knowledge of the area"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Summary: This work proposes formulating the problem of safety validation for autonomous vehicles as generating appropriate initial positions for agents to compose difficult, potentially adversarial, scenes. It proposes using a reinforcement learning approach to learn a distribution for the initial position of a pedestrian that maximizes collisions to create challenging scenarios for an autonomous vehicle. The proposed method is validated within the CARLA simulation environment. 

Clarity: The paper clearly presents an important and under-examined problem and all reviewers seemed to understand the proposed approach after reading the full paper. However, all reviewers suggest ways the paper could be reorganized to more quickly help readers grasp the core ideas. 

Significance/Originality: Reviewers had mixed feedback on the impact and originality of the work. It seems like some reviewers were bought into the problem formulation whereas others wanted to see a more experimental basis. 

Quality: All reviewers agreed that the paper has room for improvement in strengthening its premises (see “opportunities for improvement” below for more details). 

Pros: 
The paper addresses a highly critical, interesting, and under-addressed area in robotics. It was clear that all reviewers agreed this problem space is extremely important and of high interest. The paper also presents a framing of the problem that is interesting and likely to spark new discussions. 

Main opportunities for improvement: 
It seems like the paper has three main components - 1. a proposed framing of the safety validation problem as generating initial positions for simulated agents, 2. a decision to use RL to solve this problem, and 3. experimental results with a single pedestrian in a scene. 

A major contribution of the paper is that 1 is creative and somewhat unexpected, making this paper rather unique. However, at the same time, the paper has a higher burden of convincing others to adopt that same view of the validation problem. The paper would benefit from strengthening this - specifically, consider reviewer UAuu’s feedback on how to make the “point” of the paper more crisp and upfront and reviewer s5dQ’s feedback on assumptions that need to be clarified and better supported. Reviewers 7Zap and s5dQ also both comment that the results presented involve significant simplifications (e.g. single agent, simple car model). It would be worth paying special attention to clearly delineating the scope of the problem this work attempts to solve and why that problem is “big enough”. 

Regarding 2, reviewers 7Zap and s5dQ question why the proposed RL approach is needed to solve the problem given the simplifying assumptions mentioned above formulate a problem that is arguably solvable by non-ML methods. Given that using ML is a focal point for CoRL, it is necessary to clarify this point in the rebuttal. 

Finally, reviewers expressed uncertainty about whether or not the technical work is large enough to be a publishable unit or is a still ultimately a “first step”. I would recommend that the rebuttal also spend time justifying why the scope of the paper represents a complete unit of research. 

Thank you for considering our feedback, and we look forward to seeing the updated paper. 

============= Final Decision

Thank you to the authors and reviewers for the extensive and productive discussion. After discussion, the authors have agreed to incorporate key parts of the reviewers' feedback, in particular reviewer s5dQ's concerns, in the final version of the paper, and all reviewers have achieved consensus that the paper would be a useful contribution to the conference. That being said, I think reviewer s5dQ's comments about the paper's shortcomings are valid and accepting as a poster is more appropriate. 
Decision: Accept (Poster)