Learning to Reason and Act in Cascading Processes
Authors: Yuval Atzmon
Eli Meirom
Shie Mannor
Gal Chechik
Venue: Submitted to ICLR 2023 (2023)
Keywords: cascading processes
intervention
reasoning
tree search

Abstract: Training agents to control a dynamic environment is a fundamental task in AI. In many environments the dynamics can be summarized by a small set of events that capture the semantic behavior of the system. Typically, these events form chains or cascades. We often wish to change the system behavior using a single intervention that propagates through the cascade. For instance, one may trigger a biochemical cascade to switch the state of a cell, or reroute a truck in logistic chains to meet an unexpected, urgent  delivery.
We introduce a new supervised learning setup called "Cascade". An agent observes a system with a known dynamics evolving from some initial state. It is given a structured semantic instruction and needs to make an intervention that triggers a cascade of events, such that the system reaches an alternative (counterfactual) behavior. We provide a test-bed for this problem, consisting of physical objects.
We combine semantic tree search with an event-driven forward model and devise an algorithm that learns to efficiently search in exponentially large semantic trees of continuous spaces. We demonstrate that our approach learns to effectively follow instructions to intervene in previously unseen complex scenes. When provided an observed cascade of events, it can also reason about alternative outcomes.

Under review as a conference paper at ICLR 2023
LEARNING TO REASON AND ACT IN CASCADING PROCESSES DRIVEN BY SEMANTIC EVENTS
Anonymous authors Paper under double-blind review
ABSTRACT
Training agents to control a dynamic environment is fundamental task in AL. many environments, the dynamics can summarized by a small set events that capture the semantic behavior of the system. Typically, these events form chains or cascades. We often wish to change the system behavior using a single intervention that propagates through the cascade. For instance, one may trigger a biochemical cascade to switch the state of a cell or reroute a truck in logistic chains meet an unexpected, urgent delivery.
We introduce a new supervised learning setup called Cascade. An agent observes a system with known dynamics evolving from some initial state. It is given a structured semantic instruction and needs to make an intervention that triggers cascade of events, such that the system reaches an alternative (counterfactual) behavior. We provide a test-bed for this problem, consisting of physical objects.
We combine semantic tree search with an event-driven forward model and de- vise an algorithm that learns to efficiently search in exponentially large semantic trees of continuous spaces. We demonstrate that our approach learns to effectively follow instructions intervene in new complex scenes. When provided with an observed cascade of events, it can also reason about alternative outcomes.
INTRODUCTION
Teaching agents to understand and control their dynamic environments is a fundamental problem in AI. becomes extremely challenging when events trigger other events. We denote such processes as cascading processes. As example, consider set of chemical reactions in a cellular pathway. The synthesis of a new molecule is discrete event that later enables other chemical reactions. Cascading processes also prevalent in man-made systems: In assembly lines, when one task is completed, e.g., construction of gears, it may trigger another task, e.g. building the transmission system.
Cascading processes are abundant in many environments, from natural processes like chemical re- actions, through managing crisis situations for natural disasters (Zuccaro et al. 2018: Nakano et al.) 2022] to logistic chains or water treatment plants (Cong et al./ (2010). A major goal with cascading processes is to intervene and steer them towards a desired goal. For example, in biochemical cas- cades, one hopes to control chemical cascades in cell by providing chemical signals; logistics, cargo dispatch plan may be completely modified by assigning cargo plane to a different location.
This paper addresses the problem of reasoning about cascading process and controlling its qualita- tive behavior. We describe a new counterfactual reasoning setup called "Cascade", which is trained via supervised learning. At inference time, an agent observes dynamical system, evolving through cascading process that was triggered from some initial state. We refer to it as the "unsatisfied" or "observed" cascade. The goal the agent to steer the system toward a different, counterfactual, configuration. That target configuration is given as set of qualitative constraints about the end results and the intermediate properties of the cascade. We call these constraints the "instruction". To satisfy that instruction, the agent intervenes with the system at specific point in time by changing the state of one element which we call the "pivot".
To solve the Cascade learning problem, train agent select intervention given state of system and an instruction. Importantly, we operate in a counterfactual mode (See Pearl 2000). Dur- ing training, the agent only sees scenarios that are "satisfied". in the sense that the system dynamics

---

Under review as a conference paper at ICLR 2023
Figure 1: An experimental test bed Input 1: An observed cascade. for the Cascade setup. Input 1 (the unsatisfied cascade): A set of balls is
1=1.42:
1=2.04
I:3 04:
t=4
purple right-wall
purple gray pin yellow hits cyan
cyan hits gray pin
observed moving in a confined space, colliding with each other, with walls,
and with static pins (grey & black). Collisions yield a cascade of events (arrows). Input 2: A complex in- struction describes a desired "counter- factual" cascade of events and its con- straints. Output (the satisfied cas- cade): The agent intervenes and sets
Input 2:
An instruction.
Push:
purple ball
balls motion, all ftames.
Target:
yellow hits black pin
Bottleneck:
yellow hits
Aroos " highlight
Chein Count: collisions
Output:
A solution.
the (continuous, 2D) initial velocity of
t=1.75:
1=3.36: yollow hits rad
1-4.85: yellow hise black pin
the purple ball (the "pivot") achieve the goal, satisfying the constraints. Only keyframes are shown. See full videos here: https: /ttp:://youuubb be/ ulIo-2WC Sw (Anonymous)
purple ball hits the cyan ball
purple hile yellow
obey the constraints given in the instruction. The reason is that in the real world it is not possible to rewind time and simultancously obtain both satisfied and an unsatisfied sequence of events.
Steering a cascade process is hard. To see why, consider natural but naive approach to the Cascade problem: train an end-to-end regression model that takes the system and instruction as input and predicts the necessary intervention. It is challenging because in many cases, slight change in one part of the system can make a qualitative effect on the outcome. This may lead to an exponential number of potential cascades. This "butterfly effect" (Lorenz) 1993) is typical in cascading systems, like billiard ball missing another ball by thread or truck reaching a warchouse right after another truck has already left. Back to the regression approach, we empirically find that fails, presumably because the set of possible chains of events is exponentially large, and the model fails to learn how to find an appropriate chain that satisfies the instruction. We discuss other challenges in Section
Technical insights. In designing our approach, we follow two key ideas. First, instead of modeling the continuous dynamics of the system, we reduce the search space by focusing on a small number of discrete, semantic events. To do this, we design representation called "Event Tree" (Figure In a billiard game, these events would be collisions of balls. In logistic chains, these events would be deliveries of items to their target location assembly of parts. To reduce the search space, build a tree of possible future events, where the root holds the initial world-state. Each child node corresponds to possible future subsequent event from its parent. Thus, path in the tree from root to descendant captures a realizable sequence of events.
Our second idea is learn how efficiently search over the event tree. This is critical because the tree grows exponentially with its depth. We learn a function that assigns scores to tree nodes con- ditioned the instruction and use these scores prioritize the search. We also derived a Bayesian correction term to guide the search with the observed cascade: we first find the path in the event tree that corresponds the observed cascade, and then correct the scores of nodes along that path.
Modelling system dynamics with forward models. A forward model describes the evolution of the dynamic systems small time steps. There extensive literature on learning forward models from observations in physical systems (Fragkiadaki et al. (0166) Battaglia et al. 2016 Lerer et al. 2016 Watters al., 2017: Janner et al., 2019). Recent work also studied learning forward models for cascades (Qi et al. [22 Girdhar et al.] (022). However, once the forward model has been learned, the desired initial condition of the system is found by an exhaustive search. Here, we show that exhaustive search fails for complex cascades and with semantic constraints (Section (5.. Therefore, our paper focuses on learning to search not on learning the forward model. We assume that we are given a special kind of "forward" model operating at the level of semantic events. Namely, given state of the cascading system, our forward model allows to query for the next event ("which objects collide next?"), and predict the outcome of that event (velocities of objects after collision).
Test bed. We designed a well-controleed environment that shares key ingredients with real-world cascading processes. In our test-bed several spheres move freely table, colliding with each other
2

---

Under review as a conference paper at ICLR 2023
and with static pins within confined space (Figure 1 cascading process. Naturally, a simulated test-bed cannot cover the full complexity of real-world scenarios and additional research may be required. We discuss these topics in Section
The chain of collisions forms complex
Contributions. This paper proposes novel approach for learning to efficiently search for a complex cascade in dynamical system. Our contributions are: I) A new learning setup, Cascade where an agent observes a dynamical system and then changes its initial conditions to meet given semantic goal. (2) Learning a principled probabílistic scoring function over an Event Tree data structure, for searching efficiently over the space of interventions.
(3) A Bayesian formulation leveraging the observed cascade to guide the search in the event tree toward counterfactual outcome.
RELATED WORK
Learning and reasoning in physical systems. Several papers studied cascading events in the con- text of physical systems. There, the main focus was to object interactions to learn forward model from observations. PHYRE, Virtual Tools, and CREATE (Bakhtin et al., 2019: Allen et al.) 2020 Jain al. 2020) are benchmarks for physical reasoning for computer vision. They differ from our learning setup in three key aspects. First, the current paper focuses on the search problem, look- ing to satisfy a set of semantic constraints on the event sequence. Second, in the prior benchmarks, all tasks have to satisfy the same final goal, rather than being conditioned on a semantic goal. Last, their setup sequential decision reinforcement learning setup, allowing exploration, collecting rewards from the environment, and multiple retries, which are not allowed in our setup. In addition, no event-driven forward model (EDFM) currently available for these benchmarks, and training an EDFM requires additional annotations and is beyond the scope of this work. There several approaches to learn such models from temporal data, like dynamic Bayes nets (Bhattaharjya et al.
2020 Ghahramani
1998 Gunawardana & Meek 2016), which can also handle latent variables.
Allen et al. 2020) takes a Bayesian approach for updating the distribution of initial conditions given reward. We consider the underlying chain of events and update the value of the node scores in the event tree according to the observed cascade. CLEVRER, CoPhy, CRAFT, CATER, and IntPhys (Yi le al. 2020 Baradel al., 2020; Ates et al.] 2021 Girdhar & Ramanan 2020 Riochet et al.) 2018) are benchmarks for reasoning over observed temporal and causal structures in video. They differ from our setup in a few key aspects: (1) They focus on video-tracking and question answering rather
than acting (2) In CLEVRER and CoPhy, the observed cascade is available during training, which may not a reasonable assumption for real-world problems (see Section 3 (3) CoPhy estimates the value of a static observed property, like gravity, while we focus on the cascade evolution.
Roussel al. (019) studied the chain reaction problem, with a different focus than ours. Their cas- cading configuration is fully given and they study how to tune that configuration using a simulator. Our work focuses on finding cascading configuration given a partial description of it.
Graph Neural Networks have been used in physical environments (Kim & Shimanuki] 2019 Shen jet al. 2020 Bapst et al., 2019), representing the underlying state as graph, without considering temporal consequences. Temporal consequences are vital for our decision system. We propose how to transform a temporal sequence of events to a DAG.
Reinforcement learning: The "Cascade" learning setup is fundamentally different from rein- forcement learning framework (Sutton Barto 2005). The problem we try to solve is not a standard planning problem (Hafner et al. 2019), where series of actions taken sequentially. Here, an action is taken once and sets the cascade of events ("Fire and forget").
Planning in robotics: Pertsch al. (2000;; Jayaraman et al. (2019) learned from video data to predict key-frames, conditioned on a start frame and an end frame (goal). These works rely on a visual end goal. It is unclear how to them with semantic goal that includes constraints. They also rely on taking multiple actions, which is not applicable in our "Fire and forget" setup.
Causal inference: Counterfactual reasoning studied in causal inference (Pearl 2000). Most relevant is (Buesing et al. 2019) that used counterfactually augmented data for training a RL policy.
THE "Cascade" LEARNING SETUP
Cascade is a supervised learning problem. At the inference phase, The agent is provided with a dynamical system and two inputs: (1) A sequence of events called the "observed cascade" together
3

---

Under review as a conference paper at ICLR 2023
with the respective initial condition of the system. (2) An instruction that describes desired semantic properties ("constraints") of the solution. The observed cascasde does not satisfy the instruction. The agent is asked to intervene by controlling the state of one "pivot element" in the system at specific point in time. The goal is to find an intervention that makes the roll out of the dynamical system satisfy the instruction.
At the training phase, we are only given "successful" labeled examples. The "features" (x) of each label consists of (1) instruction; and (2) the initial state of the system except the controllable pivot. The "label" (y) of each sample is the initial state of the pivot, which yields the desired behavior of the system. During training we do not provide examples of failing sequences together with a successful sequence. The reason is that in reality, one cannot "roll-back" time and obtain both a failed sequence and successful sequence.
More formally, our training set D consists of N labelled ffeatures = (In, 9n). label = (v% Q())), n I...N..N N). where:
samples
D
In is the initial state of dynamical system, excluding those of its pivot element. On IRC is structured representation of an instruction.
V E y C I2 is the pivot's initial state of the solution.
Quv+) {s is a sequence of events that occur when the system is played out with the pivot initial value V/--
Al test time, a novel sample is drawn, describing an unseen dynamical system and instruction (1, g): and observed cascade roll-out (y Qyyob)) which fails fulfill the instruction. Our goal is to provide an alternative (counterfactual) initial state for the pivot element y, such that the instruction fulfilled when the system is rolled-out.
Our test bed: We introduce new simulated test bed that abstract away from specific applications. An agent observes a physical world with several moving and static objects going through a cascade of events (Figure [ top), and it is given a complex instruction "Push: purple ball " It then manipulates the direction and speed of the purple ball process that satisfy the complex set of constraints given by the instruction. In the Cascade setup, the agent is trained on a set of scenes and their goals, and is tested on new scenes and their goals.
manifesting new cascading
4 METHODS
a)
S
Solving the Cascade setup poses three major
Yoor
feasible dynamic configurations
challenges. First, our model needs to match semantic events, but simulations of dynami- cal systems typically follow fixed and small timesteps, which are indifferent to events. Sec- ond, the set of desired constraints and dynam- ical systems is compositional and large. The agent should learn generalize different sys- tems and configurations that were not observed during training. Third, wish to benefit from examples of failed cascades that are available at inference time (the counterfactual setup).
5,= [[eyan, purple)]
W
- ... >>
W. = [(,))] w. E W W e
b)
a²'
Yn
Yi
Sw - [Gyan, purple (red, black) V. (v00)) =+++
Yroat
W. ([(,)) wy Way e Yw)
Figure 2: (a) The Event Tree data structure, illus- trated according to our test-bed. S is the collision sequence of a node; Y the intervention subset of node; W is the node's world-state. See Sec- tion 4. (b) Tessellation of the intervention space.
We develop an approach that addresses these three challenges. To address the first, we de- velop a representation that focuses on key "se- mantic" events of the dynamics (e.g.. colli- sions). We build a tree of possible outcomes
such that path the tree captures realizable cascade of events. To address the second challenge, we learn scoring function that assigns values to tree nodes conditioned on the instruction. This allows us to generalize to unseen setups, and at inference time, we use the predicted node scores to search efficiently over the space of interventions, To address the third challenge, We develop Bayesian formulation that allows to integrate the "counterfactual" information with the score pre- dictions. Next, we describe each component in more detail.
4

---

Under review as a conference paper at ICLR 2023
4.1 THE Event Tree: A TREE OF POSSIBLE FUTURES
We now describe the main data structure that we use to represent the search problem the Event Tree. The event tree is designed to provide searchable data structure for realizable sequences of events. To make these searches efficient, we represent the system behavior at the semantic level. These could be any key interactions between system components, like manufacturing an item in supply chains, or protein binding the DNA to regulate the expression of a gene. Importantly. in our approach, we require that it is possible to compute the state of the system right after an event.
Each node corresponds to a sequence of semantic events. The node's children correspond to real- izable continuations of the event sequence. Namely, all possible events that could happen after the sequence S.,. We now formally describe the node properties and the expansion of the event tree.
Tree Nodes. node the event tree corresponds to the subset of interventions Yu C y that share the same prefix of semantic events. Each node has a unique prefix S, - (81, 82, .2. ..)... Specifically, after the shared prefix, different sequences of events may follow for different y E Y..
The root node describes the set possible interventions empty. Its intervention subsel is Yooot Y) See Figure 2 top.
and its sequence of events Sroot is
We define wy as the state of the system after it evolved from y E Y and yielded the sequence Su- Then, a "woord-state" of a node is defined as the set W,, {wy | by E Y.).
Given the world state of a current node, we propose an event-driven forrrad model f()). It takes as input state wy and outputs the next immediate semantic event. We parallelized it on GPU to detect possible futures for the world state W.. Appendix desscribbs the forward model in detail.
Node Expansion. Suppose each wy E W A new node u' is composed of all ((w)) that share a same next semantic event s'. Then, the event sequence of the child node u' is S. = concat s'); the intervention set is Y, = (y)Q(v) has prefix }.
decide to expand node We apply the forward model f(-) to
Expanding the tree can be viewed as tessellation refinement of the intervention space V. At each step, we pick cell and split it into multiple cells, where each child cell represents different event that occurs after a shared sequence of events, represented by the parent cell.
If the tree is fully expanded, it covers all possible futures. However, expanding the whole tree is expansive, as it grows exponentially with its depth. In the next subsection, we discuss how one can learn scoring function and use it to guide an efficient tree search.
4.2 ASSIGNING AND LEARNING SCORING FUNCTION FOR NODES
To search the tree for node that satisfies the goal, we prioritize which node to expand by learning a scoring function that assigns scores for nodes, conditioned on the instruction g. There are three key challenges in learning score function. First, we do not have ground-truth (target) scores for tree nodes, and it is unclear what would be an effective assignment of scores. Second, the training data contains only positive examples of correctly designed plans. Finally, we wish to leverage the information about the faulty observed cascade, which is only available during inference time.
To motivate our approach, consider the following naive approach to set target scores. For given tree, let the "target" u² be the node that represents the ground-truth sequence S.,. A natural choice for setting scores would be to set V(u") little guidance for searching the tree, as no signal is provided until the search hits the target node. Instead, desired property of the learning algorithm would be guide the search by assigning monotonically increasing scores along the path from the root to u².
1, and set all other scores zero. However, this provides
To address the three challenges we design a principled probabilistic approach for setting the score function. We train our model to predict the likelihood that sample from Yu, when rolled out, will satisfy the instruction g.
(I) Here, nodes on the path from the root to u² assigned monotonically increasing scores, as the tessellation gets finer and concentrates on Y,,. Additionally, this probabilistic perspective allows us to take
(u) = Pr (Q(y) satisfies gly E Yu.g).
approach at inference time prioritize nodes.
5

---

Under review as a conference paper at ICLR 2023
We use the maximum likelihood estimate of Vuu) to calculate the ground-truth scores for training. For that, take finite sample of Froot V collecting say 10 points, and use it to expand the tree. The node's score is then the fraction of the samples from Y, that reach the target node u².
V (u)= Pr y Y.- 1ys Yu.g) Pv. no.) |x,|
Nodes outside that sequence get score of O. In Section S we empirically explore alternative approaches for assigning ground-truth scores to nodes.
Counterfactual update for the score function. During inference, we observe cascade that does not satisfy the instruction, and are asked to retrospectively suggest a better solution. How can the observed cascade be used to find solution? The probabilistic score function we defined allows us to formalize this problem in a Bayesian setting. We treat the model predictions as a prior for the true score, and the information about the observed cascade as evidence. We then ask how to update the score function given the observed evidence. Formally, our goal is to solve Eq. I when it is conditioned by the evidence, (u doesn't satisfy g).
During training, our model learns to estimate the unconditioned score function V(.). In the ap- pendix, we show that we can express the Bayesian update of the scores in terms of V (uube), V(u),
V Vu|Saaaa doesn't satisfy g) V(u)- (uubr) fr(y ,y))
(3)
where fr(udbs, u) = Pr(y E Yy by E Y.) is the probability that an intervention E Y, will result in sequence with prefix Syot., It is estimated in a fashion similar to Eq. 2
model for the score function. the score function. The model takes inputs the instruction g and sequence of events S. that define the node u, and predicts a scalar score with ground -ruth labels according to Eq. (2).
Next, we describe the representation and architecture for modelling
A naive approach is to represent S, sequence. However, such representation may not convey well the relations describing the cascade of events. For illustration, in the following sequence of collision events ((A, B), (C, D), (A, E)], the collision (A,E) is driven by (A,B), because Á is com- mon for both, while (C,D) is less relevant for describing the events that lead to (A,E). Instead, we transform each sequence of events. A node in this DAG is an event that involves some elements. Each edge represents an element shared by two subsequent events. See Figure for a concrete illustration.
Directed Acyclic Graph (DAG) that captures relations in the cascade
Architecture We use a Graph Neural Network (GNN) to parameterize our score function. We represent the graph as a tuple (A, X, E) where A E {0, 1)"X" is the graph adjacency matrix, Y E Raxd is a node feature matrix, E E Rmmdd' is an edge feature matrix, and = E IRd" is a global graph feature. We chose to a popular message passing GNN model (Battaglia et al. 2018). We describe its architecture in detail in the appendix.
4.3 INFERENCE
Our agent searches the tree for the maximum scored node "MAX- Then, it randomly selects an intervention from its intervention subset y YUMAX- We consider two variants.
Maximum likelihood search: The agent performs tree search that expands the most likely nodes. At any given step, the agent stores a sorted list of nodes together with their likelihood scores, it then picks the highest scoring node from this list and expands it. The node children are then added the list with their predicted scores, and the agent resorts the list.
We limit the tree search to expand only 80 nodes, whereas in our test bed a full event tree, which contains all possible realizations, have billions of nodes, x 2.8 per unit of depth (empirically).
Counterfactual search: Here we explain how we leverage the information in the observed cascade for inference. Consider the case where the sequence of the solution is complex and the observed sequence diverges from the solution at a late point. In this case, it likely that a part of the observed chain will be informative about the solution, and will diverge at some point, To use that informa- tion, we apply the Bayesian correction (Eq. 3) term to the predicted score of every node along the observed sequence. We pick the highest scoring node, and initialize the search up to that node. Then we continue the search as described by the "Maximum likelihood search". " In practice, we trim the observed sequence, N Notserved nodes. Notserved is hyper parameer

---

Under review as a conference paper at ICLR 2023
5 EXPERIMENTS
We compared our approach to state-of-the-art baselines, including human performance. Then, we follow with an ablation study to examine the contributions of different components our approach. Next, we describe our experimental protocol, compared methods, and evaluation metrics.
5.1 A SIMULATION BENCIIMAR
[[cyan, purple), (red, black), (yellow, purple), (red, yellow). (purple, gray), (cyan, wall), (yellow, black).....
We designed well-controleed environment that share key ingredients with real-world sys- tems of cascading events. D Making it (1) sensitive to initial conditions, to allow a di- verse set of future cascades, (2) containing di- verse scenes, each describing a unique dynam- ical system. (3) including semantic goals that depend on intermediate outcomes; and (4) can benchmark counterfactual scenarios.
Yello
Pupple Cyen
Red- Black #2
Purple Yellwi B3
Ried- Yello
Purple Gray
Wall- Cyen
Vallowe Black aB
Scenes. In our test-bed, several spheres move freely on frictionless table, colliding with each other and with static pins within confined four-walled space (Figure [ Each episode de- scribes a different scene, which includes tens of collisions. Instructions.
Figure 3: Illustrates transforming a sequence of
events (top) DAG (bottom). It corresponds to the video in Figure I bottom.
structured instruc- tion describes (i) A pivot element to manipulate "Push: green ball"; (ii) A target semantic event (collision) to fulfill "Target: (iii) constraints, of two possible types. First, is "count" constraint. It resembles constraining the total amount of resources available on a logistic chain. It specifies an accumulated number of colli- sions, all the paths from the pivot the target, e.g. "Chain Count: 3 Second, is "bottleneck" constraint, which resembles a bottleneck along a logistic chain. It enforces a specific collision that lies on any path that starts from the pivot and reaches the target collision, e.g. "Bottleneck: red hits top wall". The appendix describes the instruction generation process with more examples. The task. The objective of the agent to intervene with the initial state of the scene, by setting the velocity vector of the pivot object to reach a set of collisions specified by the instruction. This often requires a precise "trick shot",, that requires detailed reasoning on how downstream events will roll out.
red hits black pin"; and
We generated dataset with ~46K scenes (we limited generation time to 80 hours), each includes 4-6 moving balls, 0-2 pins, and 4 walls and up to semantic instructions (~4.25 on average). The data is split by unique scenes, into 470 unseen scenes for test, 69 scenes for selecting hyper-parameters (val. set), and the rest are used for training. See Appendix H for more details.
Dataset:
5.2 EXPERIMENT DETAILS
Compared Methods: We compared the following methods. (1) ROSETTE (Reasoning On SE- manTic TreEs); Our approach described in Section 4 Search uses the "counterfactual" variant of the tree search (Section 4.3) by first expanding the nodes along the "observed" sequence. (2) ROSETTE-maxxL:: Like #1, but using "Maximum likelihood search" " (Section 4.3) not using the "observed" sequence. For fair comparison, of nodes total as ROSETTE--mx--L. (3) (Qi et al. 2021). The SOTA on PHYRE, using a learned forward model, noul-satisfaction classifier and exhaustive search. For fair comparison we replace their learned forward model by the full simulator of d'MMaovvvychkk et al. (202I). (4) Cross Entropy:
make that ROSETTE expands the same number
standard planner (de Boer et al. 2005 Greenberg 20.22 that optimizes the objective function
learned by compared method (3). (5) Sequential: Using sequential representation for tree chain, instead of DAG. Specifically, we represent the sequence as a graph with edges along the sequence. (Litany et al. 2022) compared recurrent versus standard synchronous propagation in GNN mod- els and found them empirically equivalent.
(6) Deep Sets regression: Embedding the instruction and the initial world state to predict continuous intervention. We embed the objects' initial posi- tions and velocities using the permutation cermutation-invariant "Deep Sets" architecture (Zaheer et al. 2017).
Examples:
link #1
link #2
link
Code and data will be released upon publication.

---

Under review as a conference paper at ICLR 2023
and use Ly loss with respect to ground-truth interventions in the "counterfactual" training sam- Sample interventions at random from an estimated distribution of ground-truth interventions. More details appear in the appendix.
ples. (7) Random:
Ablation: We also carry a thorough ablation study: First, we explore alternative approaches to label node scores along the ground-truth sequence. Linear: Linearly increases the score by V(u) depth(u) /depth("") Step: Give a fixed medium score to nodes along the sequence, and a maximal score to the target node: V(u) = 0.5+0.51..-1) (u). Dirac-Delta: Sets V (u) 1.,-(z)) (u), this baseline is equivalent to naive approach discussed in Section 4.2 Second, we compare the "Counterfactual" to the "Maximum Likelihood" search by emphasizing the former effect with a dataset that employs more complex instructions using a third constraint, and compare the two types of search across "Easy" and "Hard" instructions. We describe this dataset in the appendix. Last, we test how ablating parts of the instruction affects the ROSETTE model performance. Implementation details of the baselines and ablations are described in Section D
TREE SUCCESS
SIMULATOR SUCCESS
TREE SUCCESS
RANDOM DEEPSET REGRESSION
NA
17.6 0.3%
DIRAC DELTA
33.5 1.6%
NA
18.4 0.5%%
STEP
45.1 = 1.0%
(Q ET AL. 2021 CROSS ENTRON ROPY SEQUENTIAL
NA
21.1 + 0.9%
LINEAR ROSETTE MAX-L (OURS)
48.7 - 0.7%
NA
20.9 + 0.4%
59.7 = 0.3%
52.4
0.6%
43.1 0.3%
ROSETTE (OuRS)
60.8 = 0.3 %
48.8 + 0.3%
Table 1: Success rates. (Left) Our approach and baselines. TREE is not applicable to the first three baselines since they do not use event tree. (Right) Variants of the score function (see Ablation).
Evaluation metrics: For each episode and goal, we predict an intervention and evaluate their success rate using the following metrics. Simulator success rate: The success rate when rolling out the predicted intervention using physical simulator (Makoviy- chuk et al. 2021). This metric mimics experiment- ing in the real world. Tree success rate (where applicable): Each node in the tree represents a se- quence of events. A tree based algorithm selects a node. A "tree success" is when the selected node's sequence satisfy the instruction. This metric evalu- ates the performance of the score function and tree search, independently from errors that may be intro- duced due to the event-driven forward model.
ROSETE JOun)
Tree Success
0.65
0.65
0.45
0.2
0.4
0.15
Easy Instruction complexity
Hard
<5
5+
Count Instruction
Figure 4: Comparing "Counterfactual" search (ROSETTE) with "Maximum likeli- hood" search (ROSETTE-max for levels of instruction complexity ("Hard"" 2 or more constraints) and for two levels of "count" in- structions ("5+": more ). Using the ob- served cascade, ROSETTE performs better in complex scenarios.
We further measured refinements of these metrics by conditioning various properties of the instruction and scene. (1) Condition tree success rate on in- struction type: Unconstrained: The instruction only specifies target collisions. Bottleneck: also contains an "bottleneck" constraint. Count: contains a "count" constraint. B&C: contains both "bottleneck" and "count" constraints. (2) Condition tree success rate on complex scenarios: (2.1) Instructions with 2 or more constraints are marked as "Hard", and the rest as "Easy"; (2.2) Instruc- tions with a "count" constraint value 2 5 are considered hard. Complex scenario conditioning was evaluated on the complex instruction dataset. Using the main dataset demonstrate a similar trend (See appendix K. We report mean value and standard error of the mean across model seeds.
5.3 HUMAN EVALUATION
To assess human baseline, conducted user study with Amazon Mechanical Turk. We designed a game, where a player is given a video of the observed cascade and is asked to select one of 44 combinations of orientations (11) and speeds (4). The game is based on 30 test episodes. For com- paring with ROSETTE, we select the one (of 44) which is nearest (in L2) to ROSETTE's predicted velocity. Appendix Section describes the experiment design and further analysis of the results.
8

---

Under review as a conference paper at ICLR 2023
6 RESULTS
We first compare the performance of ROSETTE with baseline methods and human performance. We then study it properties greater depth, through a series of ablation experiments. Finally, we discuss the results of the baselines in Appendix A.I and we provide qualitative examples in Appendix B
Table [ describes the Tree and the Simulator success rates of ROSETTE and compared meth- ods. ROSETTE achieves the highest success rate for both the "Tree" success rate (60.8%) and the "Simullaed" success rate (48.8%). Achieving ~80% conversion rate from Tree to Simulated. The random baseline success rate is (17.6%), which is close to the performance of the regression model. We conjecture that the regression model fails, because can't 't "imagine" the outcomes of the input states as ROSETTE can. The Sequential approach is the strongest baseline, reaching Tree = 52.4% and Simullated = 13.1% success rates.
Next, we describe the success rate in the human study. ROSETTE achieves the highest average success rate (43.3% 1.3% vs 23.9% 2.6%). Humans displayed a large range of success rates, with the best human achieving 41.4%. while the median and worst humans were 25% and 10%. ROSETTE performed more persistent, with 46.6%, 43.3% and 40% for the best, median, and worst.
Ablation experiments: (1) Table (Right) shows the advantage of the probabilistic formulation of the score function (ROSETTE-max-I), compared to the several heuristics described in Section 5
(Righ)
The strongest baseline ("Linear") only reaches 48.7% vs. 59.7% for ROSETTE-max.
(2) Fig-
ure 4 quantifies the benefit gained by using "Counterfactual" search (ROSETTE) over Maximum- Likelihood search (Section 4.3) ROSETTE shows a relative improvement of 7.7% (45.1% vs
(A) allows an in-depth examination of the strengths and weaknesses of ROSETTE, across types of ablations, as described in Section First, we observe that the sequential baseline can find target collisions that depend on bottleneck col- lision, as well as ROSETTE. However, fails with "count" instructions (46.3% vs 60.8%), since it has no capacity for that reasoning task. Second, we observe that ROSETTE effectively uses the instruction, since any ablated part of the instruction hurt the respective success rate.
41.9%) for complex instructions. (3) Table 2
(Appendix
7 DISCUSSION
In this paper, we took a first step towards understanding how to affect a complex system of cascading events. We presented a new learning setup, called Cascade, where agent observes a cascade of events in a dynamical system and is asked to intervene and change its initial state to make the system meet a given goal. We use an event-tree representation and a principled probabilistic score function for searching efficiently over the space of interventions. We also describe an approach to counterfactually reason about an observed cascade during the tree search.
Our approach is best applied in problems that are naturally described by event-driven dynamics. As an example, consider cascading failures power grids (Schäfer af.) 2018). Here, semantic events are failures of nodes (transformers, power generators, ...) or edges (power lines). The power flow obeys known set of ODE for a given grid. When flow exceeds a powerline capacity, that line fails (an event), resulting in an effectively different grid and a different set of ODEs that govern the dynamics. The transmission system operator may wish to define goals like "no more than three failures", "no more than n people affected", "that important node must not fail". We elaborate on this use-case and other use-cases, such as logistics and evolution of natural disasters in Appendix []
One important question remains, how do studies that use our toy testhed can generalize to scenarios. We believe it can follow a similar paths as in other areas of AI where approaches mature from toy datasets to realistic problems: First, by creating benchmark dataset for a real-world domain, annotated with semantic events. Some fields have datasets that can be very natural for the problem we discussed. These may include logistics (Appendix D evolution of natural disasters and their consequences (Zuccaro al.) 2018), and cascading failures in power grids SShäfeer et al. (2018). Second, an event event-driven forward model (EDFM) needs to be trained using this dataset. Domain specific properties can be used to improve the accuracy and robustness of EDFM learned. Finally, given the EDFM, our approach can be applied.

---

Under review as a conference paper at ICLR 2023
Reproducibility Statement We provide full experimental detail about approach and baselines in Appendix D Appendix E describes the model architecture, feature represntation, and the data for training the scoring model for the event tree. Appendix HH describes the details about the data generation. Appendix [ describes the derivation and implementation of the event driven forward model. Finally, Appendix describes the details about the user study.
REFERENCES
Kelsey R. Allen, Kevin A. Smith, and Joshua B. Tenenbaum. Rapid trial- anil--nnd-eroo learning with sim- ulation supports flexible tool use and physical reasoning. Proceedings of the National Academy of Sciences, 117:29302 - 29310, 2020.
Tayfun Ates, Muhammed Samil Atesoglu, Cagatay Yigit, Ilker Kesen, Mert Kobas, Erkut Erdem, Aykut Erdem, Tilbe Goksun, and Deniz Yuret. Craft: A benchmark for causal reasoning about forces and interactions, 2021.
Anton Bakhtin, Laurens van der Maaten, Justin Johnson, Laura Gustafson, and Ross B. Girshick. Phyre: A new benchmark for physical reasoning. Advances in Neural Information Processing Systems (NeurIPS), 2019.
Victor Bapst, Alvaro Sanchez-Gone Gonzalez, Carl Doersch, Kimberly Stachenfeld, Pushmeet Kohli, Peter W. Battaglia, and Jessica B. Hamrick. Structured agents for physical construction. In JCML, 2019.
Fabien Baradel, Natalia Neverova, Julien Mille, Greg Mori, and Christian Wolf. factual learning of physical dynamics, 2020.
Cophy: Counter-
Peter W. Battaglia, Razvan Pascanu, Matthew Lai, Danilo Rezende, and Koray Kavukcuoglu. Inter- action networks for learning about objects, relations and physics. In Advances in Neural Infor- mation Processing Systems (NeurIPS), 2016.
Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalee Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Caglar Gulcehre, Francis Song, Andrew Ballard, Justin Gilmer, George Dahl, Ashish Vaswani, Kelsey Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, and Razvan Pascanu. Relational inductive biases, deep learning, and graph networks, 2018.
Debarun Bhattacharjya, Karthikeyan Shanmugam, Tian Gao, Nicholas Mattei, Kush R. Varshney, and D. Subramanian. Event-driven continuous time bayesian networks. In AAAI, 2020.
Lars Buesing, Theophane Weber, Yori Zwols, Nicolas Heess, Sebastien Racaniere, Arthur Guez, and Bean-Baptiste Lespiau. Woulda, coulda, shoulda: Counterfactually policy search. In International Conference on Learning Representations, 2019. httL::/tt::///// //ttt/ee///peneeveew httt/foruumiiddSJGDDeeee
Qiumei Cong, Wen Yu, and Tian you Chai. Cascade process modeling with mechanism-based hierarchical neural networks. International journal of neural systems, 20 1:1-11, 2010.
P. T. de Boer, Dirk P. Kroese, Shie Mannor, and Reuven Y. Rubinstein. A tutorial on the cross- entropy method. Annals of Operations Research, 134:19-67, 2005.
Katerina Fragkiadaki, Pulkit Agrawal, Sergey Levine, and Jitendra Malik. Learning visual predictive models of physics for playing billiards, 2016.
Zoubin Ghahramani. Learning dynamic Bayesian networks, pp- 168-197. Springer Berlin Heidel- berg, Berlin, Heidelberg. 1998. ISBN 978-3-540-69752-7. doi: 10.10077BF00055999 URL
https://doi. .rg. 10. 1007 1100//FF0053390
Rohit Girdhar and Deva Ramanan. Cater: A diagnostic dataset for compositional actions and tem- poral reasoning, 2020.
10

---

Under review as a conference paper at ICLR 2023
Rohit Girdhar, Laura Gustafson, Aaron Adcock, and Laurens der Maaten. Forward prediction for physical reasoning. Time Series Workshop. ICML, 2021.
Ido Greenberg, Yinlam Chow, Mohammad Ghavamzadeh, and Shie Mannor. Efficient risk-averse reinforcement learning. In Advances in Neural Information Processing Systems, 2022.
Asela Gunawardana and Christopher Meek. Universal models of multivariate temporal point pro- cesses. In AISTATS, 2016.
Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. In International Conference on Machine Learning. pp. 2555-2565 2019.
Aric Hagherg, Pieter Swart, and Daniel S Chult. Exploring network structure, dynamics, and func- tion using networkx. Technical report, Los Alamos National Lab.(LANN.), Los Alamos, NM (United States), 2008.
Ayush Jain, Andrew Szot, and Joseph Lim. Generalization to new actions in reinforcement learn- ing. In ICML, 2020.
Michael Janner, Sergey Levine, William T. Freeman, Joshua B. Tenenbaum, Chelsea Finn, and Jiajun Wu. Reasoning about physical interactions with object-oriented prediction and planning. In Proceedings of the International Conference on Learning Representations (ICLR), 2019.
Dinesh Jayaraman, Frederik Ebert, Alexei Efros, and Sergey Levine. Time-agnostic prediction: Predicting predictable video frames. In International Conference on Learming Representations, 2019. URL htttps::/openzevieet wtwpeenwwweewe.eefffeeeeeeeeeeee nwt. http://wweee...rrr.aaoao/aaaaooa
Beomjoon Kim and Luke Shimanuki. Learning value functions with relational state representations for guiding lask-and-mont motionn planning. In CoRL, 2019.
D.P. Kingma and J. Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
Adam Lerer, Sam Gross, and Rob Fergus. Learning physical intuition of block towers by example, 2016.
Or Litany, Haggai Maron, David Acuna, Jan Kautz, Gal Chechik, and Sanja Fidler. Federated learning with heterogeneous architectures using graph hypernetworks, 2022.
Edward N. Lorenz. The Essence of Chaos. UCL Press, London, 1993.
Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, and Gavriel State. Isaac gym: High performance gpu-based physics simulation for robot learning, 2021.
Ryota Nakano, Masatoshi Hirabayashi, Harrison F. Agrusa, Fabio Ferrari, Alex J. Meyer, Patrick Michel, Sabina D. Raducan, Diego Paul Sánchez Lana, and Yun Zhang. Nasa's double asteroid redirection test (dart): Mutual orbital period change due to reshaping in the near-earth binary asteroid system (65803) didymos. The Planetary Science Journal, 3, 2022.
Judea Pearl. Causality: models, reasoning and inference, volume 29. Springer, 2000.
Karl Pertsch, Oleh Rybkin, Jingyun Yang, Shenghao Zhou, Konstantinos G. Derpanis, Kostas Dani- ilidis, Joseph J. Lim, and Andrew Jaegle. Keyframing the future: Keyframe discovery for visual prediction and planning. Learning for Dynamics & Control Conference, 2020.
Haozhi Qi. Xiaolong Wang, Deepak Pathak, Yi Ma, and Jitendra Malik. Learning long-term visual dynamics with region proposal interaction networks. In ICLR, 2021.
Ronan Riochet, Mario Ynocente Castro, Mathieu Bernard, Adam Lerer, Rob Fergus, Véronique Izard, and Emmanuel Dupoux. Intphys: A framework and benchmark for visual intuitive physics reasoning. ArXiv, 2018.
11

---

Under review as a conference paper at ICLR 2023
Robin Roussel, Marie-Paule Cani, Jean-Claude Léon, and Niloy J. Mitra. Designing chain reaction contraptions from causal graphs. ACM Trans. Graph., 38(4), 2019. ISSN 0730-0301. doi: 10.1145/33064463332997,
Benjamin Schäfer, Dirk Witthaut, Mare Timme, and Vito Latora. Dynamically induced cascading failures in power grids. Nature Communications, 9, 2018.
William Shen, Felipe W. Trevizan, and Sylvie Thi ebaux. Learning domain-independen planning heuristics with hypergraph networks. In ICAPS, 2020.
Richard S. Sutton and Andrew G. Barto. Reinforcement learning: An introduction. IEEE Transac- fions Neural Networks, 16:285 22685-26,,10 2005.
Nicholas Watters, Andrea Tacchetri, Theophane Weber, Razvan Pascanu, Peter Battaglia, and Daniel Zoran. Visual interaction networks. In Advances in Neural Information Processing Systems (NeurIPS), 2017.
Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B. Tenenbaum. Clevrer: Collision events for video representation and reasoning, 2020.
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabás Póczos, Ruslan Salakhutdinov, and Alex Smola. Deep sets. NIPS, 2017.
Giulio Zuccaro, Daniela De Gregorio, and Mattia Federico Leone. Theoretical model for cascading effects analyses. International Journal of Disaster Risk Reduction, 2018.
12

---

Under review as a conference paper at ICLR 2023
A
ADDITIONAL RESULTS
Here we describe additional results and provide further discussion.
UNCONSTRAINED
BOTTLENECK
COUNT
B& C
ROSETTE (OURS)
76.5 + 0.8%
68.5 1.0%
60.8 0.7%
49.5 0.5%
COUNT
75.8 - 0.9%
68.4 0.8%
6.1
1.51
BOTTLENECK
76.4 + 0.9%
21.5 1.1%
6.21
0.77%
COUNT BOTTLENECK
76.6 + 0.8%
21.4 1.1%
6.1-0.550
4.2 0.3%
-FULL
60.8 + 0.5%
32.7 + 0.9%
11.7
6.6 0.1%
SEQUENTIAL
77.9 + 0.4%
66.9 + 1.3%
46.3 1.3%
36.5 + 1.3%
Table 2: Ablation study. In red, results that perform much worse than ROSETTE.
A.1 BASELINE RESULTS DISCUSSION
QQi et al., 2021) baseline: We believe that the (Qi et al. 2021) baseline fails for three main reasons:
(1) The baseline does not use an event-based representation. (2) It employs a classifier that is trained provide an all-or-none signal, rather than guiding the search. The ablation study (Table [ right) demonstrates the importance of guiding the search (compare "Ours" 59.7% vs "Dirac-Delta 33.5%
(3) The baseline architecture cannot reason over the temporal DAG structure of a cascade, as our GNN can. The importance of capturing the DAG structure is demonstrated when comparing "Our" (60.8%) to the SEQUENTIAL baseline (52.4%) (Table T left)
Existing Planners: We wish to provide further insight into why it is challenging to apply existing planners to this setup. The main challenge is that the optimization objective is given semantic terms about the end goal. To apply sponding objective function by training a classifier that checks the goal was achieved for a given scene and plan. Specifically, we used the existing SoTA PHYRE classifier Qi et al. (202T) A main drawback is that classifiers provide an "all or none" signal, hence fail in guiding the plan- ner through optimization. Conversely, our approach provides score (Eq. that monotonically increases through the tree, and is constructed to assist the search.
(CEM) planner, we derive a corre-
B QUALITATIVE EXAMPLES
Here we provide links to qualitative examples that we uploaded to YouTuue. They are best viewed in x0.25 slow motion. The YouTTube account we use is anonymous.
For each episode, we show a side-to-side video of the observed cascade, the ROSETTE successful case, and ROSETTE-max. failure case. The instruction is displayed on top of each video.
Push: cyan ball, Target: blue hits red, Bottleneck: cyan hits bottom wall, Chain Count: 6 In this example, ROSETTE semantically follows the first 10 collisions (in chonological order) as in the observed cascade. It then diverges from the observed cascade, making the blue hit the red. The cyan pivot comes into play already on the 1st collision, and the agent adjusts its velocity such that it shall yield the goal. The ROSETTE-max: baseline, hits the target, however it fails with the count constraint. The bottleneck collision occurs, but not on the chain from the pivot to the target. See the complete video here: https:
7 7/youtu. /ew.///CKKKCCCC//
Push: cyan ball, Target: green hits red, Bottleneck: purple hits red, Chain Count: 4
In this example, ROSETTE semantically follows the first 5 collisions (in chonological order) in the observed cascade. The cyan pivot comes into play the 3rd colli- sion. It then diverges from the observed cascade, and follows another chain of events, making the purple hit the red, and concluding with the target hit within collisions in the chain that started at the cyan ball. This task is too hard for the ROSETTE-man baseline, as it completely fails to satisfy the instruction. See the complete video here: /
be/ 4 9MmY2J
Push: green ball, Target: green hits cyan, Bottleneck: green hits purple, Chain Count: 5 In this example, ROSETTE semantically follows the first 6 collisions (in chonological
13

---

Under review as a conference paper at ICLR 2023
order) in the observed cascade. It then diverges from the observed cascade, making the green hit the purple, fulfiling the bottleneck constraint. The cyan pivot comes into play only the 6th collision, and the agent adjusts its velocity such that both the tar- get and the count constraint will by satisfied. The ROSETTE-max-I baseline, hits the bottleneck, however it fails to hit the target. See the complete video here: https:
//youtu.be/ beiteuu.b/iMedd.. AMedd_ MMedd_7Yndg 7Ynd
Push: yellow ball, Target: cyan hits red, Bottleneck: purple hits red, Chain Count:
In this example, ROSETTE semantically follows the first 5 collisions (in chonological or- der) in the observed cascade. The yellow pivot comes into play only on the 5th colli- sion, and the agent adjusts its velocity to satisfy the bottleneck constraint and the target. The ROSETTE-max-I baseline, completely fails in this task. See the complete video here: be/ Deee/LMTT6R2225
Push: red ball, Target: blue hits red, Bottleneck: red hits bottom wall, Chain Count:
In this example, ROSETTE semantically follows the first 4 collisions (in chonological or- der) in the observed cascade. The red pivot comes into play already on the 2nd collision, and the agent adjusts its velocity to satisfy the bottleneck constraint and the target. The ROSETTE-me max-ll baseline, satisfy the bottleneck but does not satisfy the target. See the complete video here: httt://cchp.///oooo.../////l/LLt
be wwt.:bbbvtvlllll.....o.....
Push: yellow ball, Target: blue hits purple, Bottleneck: blue hits right wall, Chain Count:
In this example, ROSETTE semantically follows only the first collisions (in chonolog- ical order) as in the observed cascade. The red pivot comes into play on the 3rd colli- sion, and the agent adjusts its velocity to satisfy the bottleneck constraint and the target. The ROSETTE- baseline, completely fails in this task. See the complete video here: LOY
Importantly, these examples demonstrate the usefulness of the observed cascade for tree search. ROSETTE followed the observed cascade along the part of the path that was useful to satisfy the instruction. It diverged from the path when necessary, and found solution when long cascades were essential, while ROSETTE-max-I struggled.
Finally, we note that this observation is also quantitatively supported: As we show in Figure 4 and Figure A.3] When conditioning the Tree Success rate on producing long cascades, with Chain count constraint values greater or equal to 5, ROSETTE performs 34.8 = 0.8%, while ROSETTE-max. performs at 31.3 + 1.1% showing ~ ~11.1% improvement. For Chain count values smaller than 5, they are statistically equivalent 75.1 + 0.4% and 75.4 + 0.4%.
C USER STUDY
We conducted user study with Amazon Mechanical Turk (AMT) using 30 test episodes. We designed a game where a player (rater) is given a video of the observed cascade and is asked to select one of 44 combinations of orientations (11) and relative speeds (4) (magnitude of velocity). One combination of orientation and speed was aligned with the ground-truth solution, and the rest were spaced in relation to that solution. In an offline stage, we tested which of the other orientations and speeds satisfy the goal and included those as valid solutions. We allowed the players to freely replay the observed video. We paid 1$ per game.
Figure A. shows one test episode. The upper panel provides an instruction that states the goal of that specific episode. On the left, we provide set of simple guidelines. The center panel provides the observed (failed) video. The right panel shows the initial frame, overlaid with the set of pos- sible orientations and a set of HTML. radio buttons to select the orientation and speed. The upper tab provides a set of four examples with solutions and explanations. Those examples are given in Figure A.2
To maintain the quality of the queries, we only picked users with AMT "masters" qualification, demonstrating a high degree of approval rate over a wide range of tasks. Furthermore, we also executed qualification test with a few curated episodes that are very simple. To qualify users, we made sure they do not randomly pick an answer by only qualifying users who completed episodes and had single error most. Additionally, we deleted queries from one qualified user,
14

---

Under review as a conference paper at ICLR 2023
Dample
Dampl
Cumple
coaL rectaal tiat
cauling to the parple wirtln eactly colidios shall
Instructions
gane:
Fombble
Thar atves bakk cable you sead soead
cinecion salal,
help evera, issic
some othir wish
in foo
To divettee the select color atterare froe rettooo
derial apodt visevs chanps compant cnieee
more cancee wkktttr.
ritinn swlll collo
eito
120k
Figure A.1: One test episode of the user study, See Section C for details.
who submitted answers at rate of 3-4 episodes per minute, as we qualitatively observed that should take 1-3 minutes complete episode.
Qualified users received a bonus of 0.5S, accompanied with the following message:
Thank you for doing the qualification batch for our colliding balls game. Our full study is now online. You can start doing it. Please remember to PLAY THE VIDEO and use decide about your answer. And also, take another look at the examples, as they provide more intuition about the task.
11 players have passed our qualification tests, playing 25 episodes on average. Table 3 compares the human success rate with ROSETTE and Random baseline. Showing Average, Median and Best statistics. For the Median and Best statistics, we only included users who played a minimum number of 20 episodes (8 of users).
Average
Median
Best
Random
17.6 1.1%
Humans
23.9 + 2.6%
25%
41.4%
ROSETTE
43.3 1.3%
43.3 %
46.7% %
Table 3: Success rate statistics for the user study. + error denotes the standard error of the mean (S.E.M) across the samples.
D ADDITIONAL EXPERIMENTAL DETAILS
D.1 HYPERPARAMETER TUNING
We train the model and baselines for 15 epochs. Batch size was set to 8192 to maximize the GPU memory usage. We use the Torchhh' default learning rate for Adam (Kingma & Ba 2015) (0.001). For inference, we set Noberreed to 9, the maximal tree depth to 30, we sample 10" intial states and expand 80 nodes per episode which takes ~13 seconds. The GNN uses 5 layers, with hidden state dimension 128. Hyper parameters were tuned one at a time, during an early experiment on validation set.
15

---

Under review as a conference paper at ICLR 2023
tovndiiee
rider
Car Checiio
Geunght
Deseple2
bounss
1990 Chi ITN,
12t 2310
Me
ter
paer
Ca WWrecteee St00e 100
Figure A.2: Examples provided in the user study. See Section C for details.
16

---

Under review as a conference paper at ICLR 2023
D.2 RANDOM BASELINE
We sample an intervention at random from an estimated distribution ground-truth interventions. The distribution is estimated by calculating 2D-histogram with 30 30, and approximating the distribution within each bin to be uniform.
D.3 DEEPSET REGRESSION BASELINE
Overview: For the Deepset regression baseline, we embed the instruction and the initial world state predict a continuous intervention. We use the permutation-invaaiia "Deep Sets" architecture Zaheer et al. 2017). and use an L2 loss with respect to ground-truth interventions in the "counter-
factual" training samples.
The input to the Deepset architecture (Zaheer et al., (017)) is a set of feature vectors. Each feature vector corresponds dynamic or static object in the scene. The output is a vector E IR, for predicting the controlled velocity of the pivot object.
Feature representation: Each feature vector in the set is represented by concatenation of the following fields (obj_feat(o), instruction serruutttoombbb,,,,e position, velocityl. where obj_feat(o) is defined by Eq. (4), instruction_nnt
defined by Eq. (7). position, velocity are the initial position and velocity of the object, given by the observed cascade.
Labels and loss: For ground-truth labels, we use the ground-truth velocity of the solution. We use L2 loss comparing the ground-truie truth labels with the output of the Deepset architecture.
D.4
QT AL.
2021 BASELINE
Overview: Qi2021 is the state-of-the-art approach for solving PHYRE. It uses a learned forward model, a learned goal-satisfaction classifier, and exhaustive search. For fair comparison with our analytic event-driven forward model, we replace their learned forward model by full simulator Makoviychuk et al. 2021).
Therefore, for the coal-satisfaction classifier, in each frame, we replace the set of input feature vectors coming from the region- papposaalinttaatioo---iee
(RPIN) of (Qi et al.] (021] by a set of feature vectors corresponding each object in the scene, and its kinematic state as given by the simulator. To condition the classifier on the goal, we concatenate the instruction representation to each feature vector.
Feature representation: Each feature vector of an object in a frame, is represented by a con- catenation of the following fields (obj-feat(o), instruction_ene ofj-feat (o) is defined by Eq. (4). instruction _emb, is defined by Eq. (7). position velocity, time are the respective readings from the simulator in the frame.
position, velocity, time]. where
Positive and Negative examples: For training the goal-satisfaction classifier with positive exam- ples, we use the simulation of the solution cascade. For negative examples, we use the simulation of the observed cascade.
Classifier Architecture: We use the classifier architecture of @Qi al., 2021 as provided in their public implementation, with the following adaptations: (1) We replace the RPIN representation by the simulator-driven representation described above. (2) We allow replacing the last fully connected layer by mulli-layer-perceptron (MLP) (3) We allowed more than four equally spaced input frames.
Simulator configuration: The RPIN forward model fixed timestamp model, working at seree-pe--ccood. In the full simulator used (Makoviychuk al.) 2021), we observed that does not perform well in such a coarse soarse-grained resolution, making objects to sometimes go through the walls. Therefore, we increased the simulator resolution to 10 frames-pers crames-per-second
We searched for the best srper-parameters configuration that minimizes the validation loss over the following ranges: Number of MLP hidden- -laees E [0, 1,... 6). Number of input frames E [4, 6, 10, 20), batch-size E [128, 256). Number of training epoch was set according to early stopping the validation
Hyperparam search:
17

---

Under review as a conference paper at ICLR 2023
Finally, we used the following hyper-purn ppper-prrametrrs to evaluate the model performance the test set: Number of MLP hidden-layers = 2, Number of imput frames = 4 (as in (Qi et al. 2021) paper), batch-size=122 128, Number of training epoch 17.
Evaluation: For evaluation, we randomly selected a subset of 208 episodes (10% of the test subset because inference for single episodes took -5.5 minutes.
D.5 CROSS ENTROPY BASELINE
Overview: The cross entropy method is black box optimizer for solving optimization problems. We used (Qi et al. 2021) baseline's ' classifier as our objective function. At each step, we sampled 100 points and updated the sampling distribution based on their score. We have repeated this process for 100 iterations, and chosen the highest scored intervention for evaluation. Our code is based on standard implementation Greenberg al. (022) of the cross entropy method.
For evaluation, we randomly selected subset of 208 episodes (10% of the test subset), because inference for single episodes took -4 minutes.
D.6 SEQUENTIAL BASELINE
We used the validation set to select the number layers for this baseline, E [5, 10, 20, 30]. There wasn't It any significant difference when using 5 or 10 layers, and the success rate degraded for 20 or 30 layers. Therefore, we used layers for evaluating performance on the test set.
D.7 INSTRUCTION ABLATION BASELINES
We report the "count" and "bottleneck" ablations by zeroing their respective features in the instruc- tion and using the same model weights that were used report the performance of the ROSETTE model. We did not retrain the model for these cases because the ROSETTE model was trained to handle these cases, as is evident by the "Unconstained" metric.
For ablating the "full" instruction, we retrained the model, while completely zeroing the representa- tion vector the input instruction.
E IMPLEMENTATION DETAILS OF THE MODEL OF THE SCORE FUNCTION
We use a Graph Neural Network (GNN) to parameterize our score function. We represent the graph tuple (A, X, E, where A {0, 1 is the graph adjacency matrix, Y E Raxd is node feature matrix, E mxd is an edge feature matrix, and E IR." a global graph feature. we chose to use popular message passing GNN model (Battaglia et al. 2018 that maintains learnable node, edge and global graph representations.
Architecture
The model is composed of several message passing layers, L o L where each L' updates all representations, i.e.:
x Fi+1 211 IA
Each layer L, updates the features sequentially: the node and edge features are updated by aggre- gating local information, while the global feature is updated by aggregating over the whole graph. We denote the parameters of the MLPs that are used only learnable parameters in the model. At the last layer global feature, i.e., = 1, which is then used as the score of the event node.
layer L; as 6, and note that these are the a single dimension for the
Feature representation We describe next the feature representation of the inputs to the node fea- ture matrix Y, the edge feature matrix E, and the global graph feature
We start by describing a feature representation of any of the dynamic and static objects the scene: An object feature representation, noted by obj feat(o), is concatenation of the following fields
18

---

Under review as a conference paper at ICLR 2023
obj obj-feat(o) [ome_hot(o), aaciiee, instruct.inner-ppp -pood, bottleneck.ind. count, count_ind,
(4)
where (ne_hol(o) is one-hot vector IR represented by the instruction; is islstalionary indi- cates whether the object is stationary; is is_active means that in the context a current collision, the object dynamics were coming from a collision chain that included the pivot; instruct_inne -prod the results of an inner product of one ho)(() with each of the 5 object representations at the in- struction embedding (Section H.3 Finally, bottleneck ind, count, count ind are copied from the instruction embedding.
The graph node and edge features are derived from the DAG representation (Figure + Each row of the node feature matrix Y concatenates the two objects that participate at a collision (obj_feat(obja)] (eajjj) Each row the edge feature matrix E represents obj_Jeat(o) of the object on that edge.
Last, the global feature a copy of the instruction embedding Eq.
Training data For calculating the training labels of the score function, we traverse the semantic tree along the ground-truth sequence of the solution cascade and collect the positive labels using Eq. (2). If the event tree cannot reproduce the solution sequence of a sample (due to errors accumulated by the event-driven forward model), then Eq. (2) cannot be calculated, and we drop that sample from the training set. We collect negative examples (with V = 0) by (1) taking the child nodes that diverge from the path to the ground-truth solution. (2) Traverse a random path along the tree with the same length as the ground truth sequence, and set the score of all the nodes along that path to Note that setting the scores of every node along these paths to V = is a heuristic and may introduce some label noise with respect to negative examples. Additional research may be required to analyze the label-noise consequences and address it.
F COUNTERFACTUAL UPDATE FOR THE SCORE FUNCTION
In this section, we derive the expression of the score function update according to the observed cascade (Eq. 33) We start the derivation by repeating the preliminary derivation steps introduced in the main text in more detail.
During inference, we observe a cascade that does not satisfy the instruction, and are asked retro- spectively suggest better solution. How can the information can be used to find a better solution? The probabilistic score function allows us formalize this problem in a Bayesian setting, We treat the model predictions as prior for the true score, and the information about the observed cascade as evidence. We then ask how to update the score function given the observed evidence. Formally, we condition Eq. (1) by the evidence, V(u|Saea. doesn't 't satisfy g).
We denote the set of interventions that satisfy the instruction g as Gg C C.,, and the evidence by E. Note that equivalent definition for the unconditioned score function V(.) is
V = Pr (Q(y) satisfies gly E Yu.g)
= Pr (y E G,ly (Y.))
Our evidence is that for particular Yoes- we have & Gy. Now, by definition, every y, y Yous share the same observed cascade Syots. Therefore, the evidence E can be equally formulated as y' & Gy for any y' sampled uniformly from Y Yunhe, , yf' ~ UY For brevity, we set Yous Y yhe
The conditioned score function is then,
Pr (y E Gyly ~ U (Ya), F)
We use the law of total probability and write,
19

---

Under review as a conference paper at ICLR 2023
E (
=Pr P E Pr Youu UY,);
+Pr + B., Yoia) Pr E Yohlly ~ (Y,),
=Pr E ( Pr (y E Yooely ;Y,)
+ Pr E Gglu F.) Pr (y E Youe ~ U FY
Furthermore,
Pr E Youlyy ~ U (V.), E) = Pr E E (Vu)) Pr E Yo U (Yw), E) Pr (y Y y U (K.))
As the conditioned event y E Yohe is independent of E.
The relations between node Uobe and u can be one of the three: a) the observed node is descendant of u (and therefore Yu n Yabs Y.,) b) and the observed node belong to different branches, and therefore Y, Yous = 0, or c) u is a descendant of the observed node (and therefore Y, n Yote = Yobe). However, ahs represents complete cascade rather than partial sequence, and therefore the observed node does have any children, and we can ignore c).
Let us consider each case separately.
u and the observed node are along different paths. this case,
Yu n Yobs
Y, NYC obs Y
Pr (y E Y. U (Y.))
(y Yohs |y ~ U (Y.)) = O,
and we're left evaluate Pr (y E Gyly ~ EE). Since the evidence in this case provides information about a set that is not conditioned on, is independent of y, and therefore we conclude with,
Pr (y Gyly U (Y.). E) = Pr (y Gylv U (Y.)) V(u)
is a descendant of the observed node. Here,
Ya Yods Yoes Pr (y E Yobsly ~ U (Yu)) = fr(yober ya.
In this case,
Pr E UX)))
= =Pr E
(Yse)) E) fr(yobe Mu) +Pr E GylieuuU,UUU Yos), E) (1 Sr(yose: Hw))
=Pr E G,y0~ U' E) fr Va)
+ (y e G,l ~ U (Yu n Yo.)) E) (1 fr vw))
(5)
Now,
Pr(y E Gyly ~ U (Yara) E)
= Pr E Gyly ~ U (Yose) (Vyl E Yober & fyl))
=0
20

---

Under review as a conference paper at ICLR 2023
Since the evidence indicates that for every s' Yobs the resulting sequence Syobs does not satisfy the goal. Furthermore,
Pr (y E G ~ U (Yu n 2o B) = Pr (y E Ggly ~ U (Yu n Yob))
(6)
As E' does not add information when we sample from (Yu n o.a).
Therefore.
Pryyeg,y~UU - E) Pr(y (y e Ggly
fr(yobss Uu))
Now
P U (r 9y 9 (oba)) (r(yaossVbe
- Py G Ys. (1 fr(yobe: Bu))
Namely,
V(u) = V(uobs)
fylyobs, Uu) + Pr (y E Gyly U (Ya n Y.. (1 fr(yobss Bal)
or
(y Gyly T V(u) ((obb)) fr(yooer Ua)
fr(yobs Vu)
Plugging this back to Eq. we obtain:
Pr (y Ggly U (Yoba), , E) -V( V(uors) S Bu)
which is our final result.
G RELATION TO CAUSAL-INFERENCE
The DAG representation (Section is useful for graphically representing instance of cade, but we intentionally avoid naming a Causal DAG, because it can't represent dependencies between events that are explicitly observed in the video. E.g., in the example [(A, B), (C. D), it may be that (A,E) depends on (C,D) because C blocks D from reaching E before A do. The event tree can simulate this behaviour, while the DAG (C,D) (A.E)) is unaware of it. From a formal causal inference perspective (Pearl 2000), our event tree is the part of our approach that can be related the formal "Structured" Causal Model (SCM). As it is generative model that reflects the data generation process; it can account for complex dependencies between events; and every edge corresponds a function, namely, the event-driven forward model.
(A, E) in Section
H DATA GENERATION DETAILS
H.1 VIDEO GENERATION DETAILS
In this section. we describe the generation process of the dynamical scene. We first create an "un- perturbed" video. Then, we perturb the video by modifying the velocity of specific element, which will be later designated as the pivot. We let the perturbed video roll out, validate that it is indeed semantically different than the unperturbed video, and label as the "observed" video. The unper- turbed video now be used as reference for our instruction generation process. It is realization of specific, complex, semantic chain of events that both semantically different than the perturbed ("observed") video and is also feasible, e.g, by setting the intervention value revert the pertur- bation. This flow guarantees that we can ask meaningful instructions on the "observed" that are guaranteed be realizable.
21

---

Under review as a conference paper at ICLR 2023
The unperturbed video. We construct the unperturbed video by iteratively adding spheres and collisions in a physical simulator (IsaacGym (Makoviychuk et al. (022I) increasing the video com- plexity. We start by placing sphere in the confined four-walled space and assign it random velocity.
The dynamics of sphere moving freely in confined square area be expressed analytically. We pick a random time fj, hitting velocity, and hitting angle for the first collision. We analytically solve for the initial position and velocity at 0 that will result in the a collision at 4 with the specific hitting velocity and angle. We assign these value to a randomly colored sphere.
Due to discrepancies between the simulator dynamics and the kinematic analytic model, roll out the dynamical system in the simulator, and record the system state immediately after a collision.
We continue adding spheres iteratively. Given a state at 4, we randomly select sphere O₂ from the existing spheres, collision time 1,+1, hitting angle and velocity. We solve analytically and find the initial position and velocity at to that will result in a collision with O corresponding parameters. We roll out the dynamical system, and update the velocities and positions records after each collision with the empirical values from the simulator.
Our simple kinematic model assumes the target sphere and the newly added move freely. However, other spheres may cross their trajectories, resulting in an a collision that will distract the spheres from their designated path. However, this simply means that the planned random collision was replaced by a different collision. Since we update our records of the resulting collisions and corresponding output velocities and positions using the simulator, this does not pose any serious limitations.
The observed video. We pick a random sphere from the set of spheres and assign it different velocity at t = O. We roll out the system in the simulator and log all resulting collisions. We validate that the resulting collision sequence is different than the unperturbed video collision sequence. We now have two videos that differ only in the initial velocity of specific sphere, but result in substantially different semantic chain of events.
H.2 INSTRUCTION GENERATION DETAILS
We describe the instruction generation process when given an "observed" video, and a "counterfac- tual" video that displays an alternative cascade of events.
Given a ground-truth video, its sequence of collisions, and a pivot, we randomly sample an in- struction: Starting by randomly sampling a target collision from the sequence. And then, we ran- domly sample up to two constraints that accompany the goal. For constructing the constraints, first represent the sequence of collisions using a DAG, in a similar fashion as described standard NetworkX functionality (Hagberg (1) We use "dag.anncestors(" to get a list nodes for the "bottleneck" constraint.
Fig- al., 2008) for graph traversal:
ure 3 then
(2) We use "all simple.ppthss)" count the nodes in a chain reaction between the pivot and the target collision.
To avoid trivial goals, we drop an instruction if it is fulfilled by the observed video (rather than the "counterfactual" video). We sample up to unique instructions for each scene (~4 average).
H.3 INSTRUCTION FEATURE REPRESENTATION
We assume perfect lexical perception, and provide the agent with the a structured vector representa- tion of each instruction, by concatenating the following fields:
instruction_embl [largel.objar aurgel_obj. b, pivol_obj, boltleneck..ojj. .a,
bottleneck.oke .bjj b, bottleneck ind,, count, count_ind,
(7)
where target_obj.a parget_obj_l Jb are the object representations of the target collision. pivot_obj represents the pivot. Noftleneck_obj) _obj_, bottleneck_oby bb, bottleneck. .ind represent the two "bot- tleneck" objects and a binary indicator scalar. If an "bottleneck" constraint is not applicable for an instruction, we them all to count, count_ind are 2 scalar values: One for the number of collisions of the chain "count" constraint, and another used binary indicator for the "count" constraint. Similarly if a "count" constraint is not applicable for an instruction, we set both count and count ind to O.
22

---

Under review as a conference paper at ICLR 2023
Finally, note that each object is represented by one-hot vector E R!2, because the environment has 12 types of unique objects: 6 colored balls, static pins, and 4 walls.
H.4 COMPLEX INSTRUCTIONS DATASET
For the complex instructions dataset, we add a third object centric constraint that counts the number of interactions specific object makes the paths from the pivot the target collision. It resembles constraining the amount of resources available per instance on a logistic chain. With an additional constraint
test approach more challenging task that has a large variety of instructions that have 2 or more constraints. We split the evaluation set to "Hard" instructions that have 2 or more constraints, and "Easy" instruction with 0-1 constraints. We generated instructions for the same scenes as in the main dataset, which yields -4.5 instructions per scene. The test set consists of 2190 episodes, where 54% are "Hard" instructions.
I THE FORWARD MODEL
In our physical setup, the dynamics are prescribed by the position and velocity of = (pos), vel)), = 1...n of each of n objects in the environment. The world state wº of node is then a tuple
w/ .f....)..
where for the root node 0 for all rz.
The forward module takes as input a world state we it outputs the next semantic event (s'), ) and a state [wx) w/-
"'?,'d?,..."" 4)) immediately after the predicted semantic event at t, The section divided into three parts. First, we describe the analytical equations that control if two objects will collide. Then we show how can leverage the analytic model to efficiently branch out from node in the event tree. Finally, we fill in the missing details and present the full forward model.
The collision detector. Assume two spheres = a, 3 moving freely on a plane with an initial velocity of v; and position r; f O. Each sphere has radius of If the two sphere collide, then, at the moment of collision, the spheres intersect at single point. We can use simple geometric calculation to find their planar distance. The distance between the center spheres is la + +lg, while the vertical distance between the two centers is 1/a (s). The resulting planar distance is then:
d == V(l + t1)2 (la - 11)² = 22/
Therefore, in order check if the spheres collide, check if the planar distance between the two spheres ever equal to d,
|r(]||== ||r. + 1- Vy = d²
(10)
This is a quadratic equation in 4, which we can solve for analytically. If the discriminant is non- negative, the collision time corresponds to the smaller root. The spheres" velocities immediately after the collision are given by:
2m2
(V1 - V2. - Y2)
V
=
V1
(y1 y2)
(11)
Mi
m2
|ly1 - y2ll"
2m2
(V1 V1 y2)
V/2
V2
(y2 M1)
(12)
mi + m2
|lyı y2ll
Likewise, is trivial to obtain an analytical expression for the collision time and output velocity of a collision between freely moving sphere and each of the static walls bounding the spheres (should the collision occur). The sphere's 's velocity in the direction orthogonal to the walls flips, while the parallel velocity remains the same.
23

---

Under review as a conference paper at ICLR 2023
Parallelizing collision detection. The collision detector provides an analytic condition that vali- dates whether a specific collision occurs.
(Ar Av)² 4(|Ar|| - dv] > 0
(13)
Eqs. 9 13 can be solved in parallel for multiple tuples of (r1, V2, V2) on a GPU using packages
such as Py Torch. Given an intervention set of Yu, and a corresponding world-state set W., u. we iterate over all possible collisions S.y = (O,, O₃). For each collision between object and j our collision detector by extracting the corresponding coordinates cy cg byr from E W.. (Eq 8 We can do in parallel for all world states w Wa. If collision is predicted, we construct a new node child u' of u. We associate with it the interventions for which the collision detector returned non-null time for the collision, Y", the corresponding set of post-collision world state W and the event sequence S concat Su, (O,, Oÿ))
apply
The complexity is quadratic in the number of object rather than linear in the number of interventions. This allows apply our algorithm with high number of interventions, and therefore enable us to consider delicate sequences collision that would require refined "trick shots".
This approach considers every two objects Oi, O2 as moving freely. However, another object in the environment, e.g, O2, may interact with O (without loss of generality) before the collision. This necessarily means that the collision time f, precedes laj. In order account for this, we hold an additional structure that maintains the minimal collision time for every w E W., We update it as we iterate over all possible collisions. Then, we associate each w W, and its corresponding y e Y. to the event node corresponding to the collision with the earliest collision time.
J ADDITIONAL SETUPS
Here, we present examples for additional setups for which our formalism can be applied.
J.1 LOGISITICS
While logistics is a complex field, we describe simple model that captures the essential components of logistics problem.
Consider a large logistics enterprise that needs to coordinate shipping from multiple locations. The enterprise has multiple carriers (e.g, trucks or airplanes) v, i 1.mm and routes them between logistic centers at r = I..n.
A plan is schedule for each carrier Ty. where a schedule Ty is sequence of arrivals and departures between various logistic centers,
= [(r) co Pou)). ) (r), t +1 to.))...)
Not all plans are feasible. Each carrier can travel at a range of velocities, resulting in range of arrival times to the possible destinations. Carriers can exchange cargo is they are present at the same logistic center.
Now, assume a logistic center is suddenly shut down. Rescheduling all carriers is unfeasible, some may be already committed to a route, or may not easily diverted (e.g., are airborne). Furthermore, recomputing new plan for the complete enterprise might be computationally heavy. Finally, it seems reasonable that re- re-planning of only the routes carriers that were suppose to arrive to the closed logistic center may be enough. We denote those & carriers the rescheduled carriers. Note that while only some of routes may be re-planned, other carriers might affected as well due to a cascade of delays even cargo exchange cancellations.
Such re- re-planning may be constrained by semantic instruction. For example: "Carrier X should only make two deliveries". "Carrier Y should meet carrier Z before meeting Carrier W". etc.
We now cast this problem into our general framework. described in Section 4 An event is the arrival or departure of carrier logistic center. Each intervention y E 3 is set of plans the rescheduled
24

---

Under review as a conference paper at ICLR 2023
earriers,
y = (To, Thy... Tk))
world state wy is the position of the & carriers at different time ((D) E))). (pf.......55 (p)..)) The forward model takes as input world state and outputs all world state that obey the following two rules: 1) At least one carrier moved constraints. If carrier i can move at velocity range [ummn: mmadl and it moves between two logistics centers Ta and Tb, then the transition time must be in
different logistic center. 2) The transition of carriers follow physical
||r rbll
|ra rbll
(14)
Umin
For simplicity, assume that there is cargo limit. The expressions for the induced probability, Eqs. remains the same.
J.2 FAILURE CASCADES IN POWER GRIDS
Cascading failures in power grid may cause large blackout with substantial economical damage Schäter et al. (2018). Cascading Power failures may be induced due to random fluctuations and can develop on orders of seconds. Human operators or complex control mechanism may not be able react in time. The transmission system operator may use event-driven forward model to find fast automated reactions for unseen dynamical configurations to avoid cascading failures.
Here, semantic events are failures of nodes (nodes; e.g., transformers, power generators, etc.) or power lines (edges). Power flow follows a known set of ODE for given grid (eqs 14-15 Schäfer
al. 2018)):
d
0, Wi+
(15)
dt
d I P. nWw +
Kiy sin (0, 0,),
(16)
dt
where, 0,, wi are the dynamical variable at node i, Pt is the power input (or output) at node i, and Kej is a weighted adjacency matrix representing the grid connectivity. If at some point in time the flow Fij exceeds the powerline capacity akij, E [0, 1] (eqs 1-2), the line fails. This condition can be formally written
Fij (t) Kip sin (0g (f) 0, (t)) > aKij.
If the line fails, the dynamics are governed by effective coupling matrix K,j, and the dynamics in Eqs. 16] accordingly.
A failure of node may induce outage to region. The transmission system operator (TSO) might define goals such as "no more than three failures", "the maximal number of affected people should be less than n", "these highly important nodes should not fail" etc.
J.3 EVOLUTION OF NATURAL DISASTERS
Finally, another use case is the evolution of natural disasters. Zuccaro et al. (0188 provides a full description of an event tree. It models transitions between events like a "seismic shock" which can lead to "landslide" and result with "traffic accident", and how taking preventive measures like "evacuate population" can influence the total damage caused by the crisis.
K COMPLEX SCENARIO CONDITIONING FOR THE MAIN DATASET
In Figure A.3 we provide the results for complex scenario conditioning for the main dataset (with two type of constraints). The results demonstrate a similar trend in the complex instruction dataset in Figure
25

---

Under review as a conference paper at ICLR 2023
ROSETTE (Ours)
0.7
0.75
ROSETTE-maxi
Tree Success
0.65
0.7
0.5
0.35
0.45
0.3
Easy Instruction complexity
Hard
<
5+
Count instruction
Figure A.3: Comparing "CCunterfactual" search (ROSETTE) with "Maximum likelihood" search (ROSETTE-max-I) for 2 levels of instruction complexity ("Hard": or more constraints) and for two levels of "count" instructions ("5+": 5 or more ). Here we use the main dataset. Using the observed cascade, ROSETTE performs better in complex scenarios.
26


Reviews:
[{"review_id": "lcUscWC419W", "reviewer": "Reviewer_bYuU", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies how to intervene in a dynamic system of cascading events, where events might trigger other events and lead to the “butterfly effect” in the end. Specifically, they propose a relatively simple simulation environment as a testbed for this problem - an agent observes a physical world with several moving and static objects and then manipulates the direction and speed of the purple ball to satisfy the complex set of constraints given by the instruction. They define the problem as a supervised learning setup, and then propose a method that uses tree search for modeling the forward process and employs machine learning to learn to search effectively in the exponential space. They present the superior performance of their method compared to several baseline models. They also provide various ablation studies and a human evaluation to demonstrate the effectiveness of their method. ", "summary_of_the_review": "The paper studies a new problem of intervening cascading process and proposes a supervised learning setup with a simulation environment as well as a better method for this problem. The paper looks convincing to me overall although I might not be very familiar with the literature of this work. I am leaning toward acceptance for this paper.", "strength_and_weaknesses": "Although I am not an expert on this problem of modeling cascading process, this paper is generally well-written and understandable to me. The following reviews are from the perspective of a person with general machine learning background.\n\nStrengths:\n1. This paper provides a precise formulation of the cascading process and the problem setup of making interventions in such systems.\n2. They provide a testbed for this problem. Despite being simulated, it's a decent setup that people can follow and study the problem in a controlled and plausible way.\n3. The method they proposed looks reasonable to me. They also compare with a bunch of baselines, some of which were proposed recently. So, the comparison looks convincing.\n4. Regarding the experiments, they provide relatively comprehensive ablation studies and demonstration cases, which I think verifies their claims.\n\nWeaknesses:\n1. The scope of the cascading process can be very large, while the paper only focuses on a relatively simple simulation environment. It would be better to avoid overclaiming in the title or introduction.\n2.  Although using the simulation environment is a reasonable setup right now, it's unclear how the studies in this toy setup can generalize to real-world scenarios. It might worth discussing at the end of the paper.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "SevpUEqXM1", "reviewer": "Reviewer_1vrT", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a\n\n1. A new learning setup called Cascade in which the agent is provided a semantic goal and a failed trajectory and the goal of the agent is to predict an intervention for the initial state of a pivot object so that the semantic goal is achieved counterfactually.\n2. A new dynamics model based on semantic events.\n3. A method to efficiently search or plan using the event-based dynamics model.\n    1. A probabilistic scoring function\n    2. Leveraging observed cascade to guide search.", "summary_of_the_review": "The general idea of an event-based world model and the idea of planning in it is very interesting. But I feel that the paper is taking too many steps simultaneously and thus making it difficult to strongly claim any one contribution. For instance, it is hard to judge the value of event trees if the evaluation is done on a new benchmark also proposed in the same paper. Similarly, it is hard to judge the value of event trees if the planning is done by a newly proposed planning algorithm rather than some standard and well-known planner.", "strength_and_weaknesses": "### Pros\n\nThe idea of an event-based state space, an event-based world model and the idea of doing planning inside it is very interesting. \n\n### Weaknesses/Questions\n\n1. Why not use an existing benchmark like PHYRE/TOOLS if the focus is on proposing a new planning framework? Are event trees not applicable to PHYRE/TOOLS benchmarks? If not, then perhaps the proposed approach is too tailored to the proposed benchmark.\n2. Could we apply a standard planner like CEM on event-trees?\n3. Very little information about the forward model is provided. Can event-based world models be learned directly from pixels similarly to how we learn latent-variable temporal generative models e.g. RSSM [1]?\n\n[1] Hafner, Danijar et al. “Learning Latent Dynamics for Planning from Pixels.”", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "L4TIbb_BOP", "reviewer": "Reviewer_b7q6", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a new learning setup called cascade in which an agent observes a dynamical system and then changes its initial state’s conditions to achieve a desired goal. The paper also proposes an interesting approach for learning a probabilistic scoring function over an “event tree” data structure to efficiently search for a complex cascade in a dynamic system. Bayesian formulation is used to leverage the observed cascade to guide the search. Experimental results show the usefulness of the proposed approach.  ", "summary_of_the_review": "The paper presents an interesting and new learning setup with a novel learning approach. This seems like an exciting direction for the research community. ", "strength_and_weaknesses": "Strengths: \n1. The paper is well written in terms of  introducing the new setup with good context. \n2. The proposed learning setup, using event trees for the search space, and the scoring method – all seems very novel when combined together. \n\nWeaknesses: \n1. It would have been more interesting to see results on one more benchmark.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject