Baby Intuitions Benchmark (BIB):  Discerning the goals, preferences, and actions of others
Authors: Kanishk Gandhi
Gala Stojnic
Brenden M. Lake
Moira Rose Dillon
Venue: NeurIPS 2021 Poster (2021)
Keywords: Cognitive Psychology
Common Sense
Reasoning
Cognitive Development
Intuitive Psychology

Abstract: To achieve human-like common sense about everyday life, machine learning systems must understand and reason about the goals, preferences, and actions of other agents in the environment. By the end of their first year of life, human infants intuitively achieve such common sense, and these cognitive achievements lay the foundation for humans' rich and complex understanding of the mental states of others. Can machines achieve generalizable, commonsense reasoning about other agents like human infants? The Baby Intuitions Benchmark (BIB) challenges machines to predict the plausibility of an agent's behavior based on the underlying causes of its actions. Because BIB's content and paradigm are adopted from developmental cognitive science, BIB allows for direct comparison between human and machine performance. Nevertheless, recently proposed, deep-learning-based agency reasoning models fail to show infant-like reasoning, leaving BIB an open challenge.

Baby Intuitions Benchmark (BIB): Discerning the goals, preferences, and actions of others
Kanishk Gandhi New York University
Gala Stojnic
Brenden M. Lake New York University
Moira R. Dillon New York University
New York University
Abstract
To achieve human-like common sense about everyday life, machine learning systems must understand and reason about the goals, preferences, and actions of other agents in the environment. By the end of their first year of life, human infants intuitively achieve such common sense, and these cognitive achievements lay the foundation for humans' rich and complex understanding of the mental states others. Can machines achieve generalizable, commonsense reasoning about other agents like human infants? The Baby Intuitions Benchmark (BIB) challenges machines to predict the plausibility of agent's behavior based on the underlying causes of its actions. Because BIB's ' content and paradigm are adopted from developmental cognitive science, BIB allows for direct comparison between human and machine performance. Nevertheless, recently proposed, deep-learning- agency reasoning models fail to show infant-like reasoning, leaving BIB open challenge.
Introduction
Humans have rich capacity to infer the underlying intentions of others by observing their actions. For example, when we watch the animations from Heider and Simmel (1944) (see video? and Figure 1), we attribute goals and preferences to simple 2D shapes moving in a flat world. Using behavioral experiments, developmental cognitive scientists have found that even young infants infer intentionality in the actions of other agents. Infants expect agents: to have object-basee goals (Gergely et al., 1995; Luo, 201 1: Song et al., 2005; Woodward, 1998, 1999; Woodward and Sommerville, 2000); to have goals that reflect preferences (Repacholi and Gopnik, 1997; Kuhlmeier al., 2003; Buresh and Woodward, 2007): to engage in instrumental actions that bring about goals (Carpenter et al., 2005; Elsner et al., 2007: Hernik and Csibra, 2015; Gerson al., 2015; Saxe et al., 2007; Woodward and Sommerville, 2000); and to act effi- ciently towards goals (Gergely et al., 1995; Gergely and Csibra, 1997, 2003; Liu et al., 2019, 2017; Colomer et al., 2020).
Figure 1: A still from Hei- der and Simmel (1944). De- spite the simplicity visual display, we ascribe intentionality to the three shapes in the scene: The large triangle chasing the small triangle and circle, whose goals are to avoid it.
Machine-learnine and AI systems, in contrast, much more limited in their understanding of other agents. They typically aim only to pre- dict outcomes of interest (e.g., churn, clicks, likes, etc.) rather than to learn about the goals and preferences that underlie such outcomes. This impoverished "machine theory of mind"2 may be critical difference between human and machine intelligence more generally, and addressing it is crucial machine learning aims to achieve the flexibility of human commonsense reasoning (Lake al., 2017).
'The dataset and code available here: https://aanihhkgandii.com https://www.youtubb. wwwp:uuuee.coooeeeee homm/watch?ve. wwt.///aaaaa..////aa...000020 TVN
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Recent computational work has aimed to focus on such reasoning by adopting several approaches. Inverse reinforcement learning (Ng et al., 2000; Abbeel and Ng, 2004; Ziebart et al., 2008; Ho and Ermon, 2016; Xu et al., 2019) and Bayesian approaches (Ullman et al., 2009; Baker et al., 2009, 2011, 2017; Aara-Ettinger 2019) have modeled other agents as rational, yet noisy, planners. In these models, rationality serves as the tool by which to infer the underlying intentions that best explain an agent's observed behavior. Game theoretic models have aimed to capture an opponent's thought processes in multi-agent interactive scenarios (see survey: Albrecht and Stone, 2018 8), and learning- based neural network approaches have focused on learning predictive models of other agents' latent mental states, either through structured architectures that encourage mental-state representations (Rabinowitz et al., 2018) or through the explicit modeling of other agents' mental states using a different agent's 's forward model (Raileanu et al., 2018). Despite the increasing sophistication of models that focus on reasoning about agents, they have not been evaluated or compared using a comprehensive benchmark that captures the generalizability of human reasoning about agents. For example, some evaluations of machines' reasoning about agents have provided fewer than 100 sample episodes (Baker et al., 2009, 201 1, 2017), making infeasible evaluate seaninggbbssed approaches that require substantial training. Other evaluations have used the same distribution of episodes for both training and test (Rabinowitz et al., 2018), making it difficult to measure how abstract or flexible a model's performance is. Moreover, existing evaluations have not been translatable to the behavioral paradigms that test infant cognition, and their results cannot be analyzed in terms of the representations and processes that support successful human reasoning.
Our benchmark, the Baby Intuitions Benchmark (BIB). presents comprehensive set of evaluations of commonsense reasoning about agents suitable for machines and infants alike. BIB adapts experi- mental stimuli from studies with infants that have captured the content and abstract nature of their knowledge (Baillargeon et al., 2016; Banaji and Gelman, 2013). It provides a substantial amount of training episodes in addition to out-of-distribution test episodes. Moreover, BIB adopts a "violation of expectation" (VOE) paradigm (similar to Riochet et al. (2018); Smith et al. (2019)), commonly used in infant research, which makes its direct validation with infants possible and its results interpretable in terms of human performance. The VOE paradigm, moreover, offers an additional advantage relative to other measures of machine performance like predictive aceuracy, in that it reveals how an observer might fail: VOE directly contrasts one outcome, which requires high-level, human- like understanding of an event, to another one, which instantiates some lower-level, heuristically, or perceptually based alternative. BIB thus presents both a general framework for designing any benchmark aiming to examine commonsense reasoning across domains as varied as agents, objects, and places, as well as key step in bridging machines' impoverished understanding of intentionality with humans' rich one.
AGENT (Shu et al., 2021), a benchmark developed contemporaneously with BIB, is inspired by infants' knowledge about agents and has been validated with behavioral data from adults. Similar to BIB, AGENT challenges machines to reason about the intentions of agents as the underlying cause of their actions. Both benchmarks test whether models can predict that agents have object-based goals and move efficiently to those goals. There are nevertheless key differences between BIB and AGENT. First, BIB evaluates whether models can reason about multiple agents, inaccessible goals, instrumental actions, and the differences between the intentions of rational and irrational agents; AGENT does not test these competencies. The advantage of including them is that they introduce additional blements-beyond a single rational agent and one or two possible goal objects- that models must flexibly account for in their reasoning. These competencies, moreover, extend the infant cognition literature, potentially allowing BIB to further inform tests for infants. Second, BIB and AGENT evaluate new models differently. AGENT involves training on many different leave-out splits, where those splits include relatively minor differences between the training and test sets. BIB, in contrast, presents a single canonical split designed to maximally evaluate the abstractness of model's reasoning: Models tested BIB must flexibly combine learning from different types of training scenarios to solve a novel test scenario. We thus ultimately see BIB and AGENT as complementary and hope that new models focused on commonsense reasoning about agents will be evaluated on both.
NNote that in the psychology literature, "theory of mind" typically refers the attribution of mental states, such as phenomenological or epistemic states (i.e., perceptions or beliefs) to other intentional agents (Premack and Woodruff, 1978). In this paper, we address on only one potential component of theory of mind, present from early infancy. which focuses on reasoning about the intentional states, not the phenomenological or epistemic states, of others (Spelke, 2016).
2

---

2 Baby Intuitions Benchmark (BIB)
BIB focuses on the following questions: 1) Can AI system represent an agent having preferred goal object? 2) Can it bind specific preferences for goal objects to specific agents? 3) Can understand that physical obstacles might restrict agents' actions, and does it predict that an agent might approach a nonpreferred object when their preferred one is inaccessible? 4) Can it represent an agent's 8 sequence of actions as instrumental, directed towards higher-order goal object? 5) Can it infer that a rational agent will move efficiently towards a goal object?
Following the VOE paradigm, each of BIB's tasks includes familiarization phase and test phase, together referred to as an "episode." The familiarization phase presents eight successive trials introducing the main elements of the visual displays used in the test phase and allows the observer to form expectations about the future behavior of those elements based on their prior knowledge or learning. The test phase includes an unexpected and expected outcome based on what was observed during familiarization. Typically, the unexpected outcome is perceptually similar to the familiarization trials but is conceptually implausible, while the expected outcome is more perceptually different but involves no conceptual violation. The unexpected outcome is thus unexpected only if the observer possesses an abstract understanding of the events, and the expected outcome reflects lower-level, heuristically, or perceptually based alternative. When VOE is used with infants, their looking time to each outcome is measured, and infants tend to look longer at the unexpected outcome (Baillargeon et al., 1985; Turk-Browne et al., 2008; Oakes, 2010).
Inspired by Heider and Simmel (1944), the primary set of visual stimuli present fully observable "grid world," shown from an overhead perspective, and populated with simple geometric shapes that take different roles (e-g- "agents," "objects," "tools") and provide few cues those roles. We chose this type of environment as particularly suitable for testing Al systems (e.g., Baker et al., 2017: Rabinowitz al., 2018) because it allows for procedural generation of large number of episodes, and the simple visuals focus the problem on reasoning about agents. This design will also allow infancy researchers test new questions about infant's 's understanding of agents in future work. While BIB and the baseline models tested here focus on this 2D grid-world environment, we have also instantiated the stimuli in 3D as means of varying perceptual difficulty in future studies evaluating other models (appendix A).
2.1 Can an Al system represent an agent as having a preferred goal object?
Developmental Background. Infants infer that agents have preferences for goal objects, not goal locations (Gergely et al., 1995; Luo, 2011: Song et al., 2005; Woodward, 1998, 1999; Woodward and Sommerville, 2000). As illustrated Figure (left), Woodward (1998)'s seminal study showed that when 5- and 9-month-old infants saw a hand repeatedly reaching to ball on the left over bear on the right, they then looked longer when the hand reached to the left for the bear, even though the direction of the reach was more similar in that event the events in the previous trials. These results suggest that the infants expected that the hand would reach consistently to a preferred goal object as opposed to preferred goal location. Other studies have shown that infants' interpretations restricted to reaching events. For example, infants attribute a preference for goal objects to a 3D box during live puppet show when that box seemingly exhibits self- self- propelled motion. (Luo, 201 1; Luo and Baillargeon, 2005; Shimizu and Johnson, 2004). When shown an agent repeatedly moving to the same object approximately the same location, do machines infer that the agent's goal is preferred object, not location?
not
Familiarization Trials. The familiarization shows agent repeatedly moving towards specific object in a world with two objects (Figure 2a center). The agent's 's starting position is fixed, and the locations of the objects are correlated with their identities such that the preferred object and nonpreferred object appear in generally the same location across trials (appendix Figure 9 and 10).
Test Trials. The test uses two object locations that had been used during one familiarization trial, but the identity of the objects at those locations has been switched. In the expected outcome (Figure 2b). the agent moves to the object that had been their goal during the familiarization. i.e., their preferred object, but the trajectory of their motion and the location of that object is different from familiarization. In the unexpected outcome (Figure 2c), the agent moves to the nonpreferred object, but the trajectory of their motion and the location they move to is the same as fumiliarization. The model is successful if it expects the agent to go to the preferred object in different location.
3

---

FundBirraatiinn craad)
(e) Test: Expected
(b) Thst: Expacied
Unexpected
Unxpected
Figure 2: Can machines represent agent's preferred goal object? Inspired by Woodward (1998)'s study with infants (left), BIB presents an agent navigating to their preferred goal object in approximately the same location across eight familiarization trials (a). At test, the location of the preferred goal object changes. The expected outcome (b) presents the agent moving their preferred goal object in a new location, and the unexpected outcome (c) presents the agent moving to their nonpreferred object the preferred object's old location. This evaluation been rendered in 2D (middle) and 3D (right).
Figure 3: Can machines infer that rational agents efficiently towards their goals? Inspired by Gergely et al. (1995) (left), BIB presents a rational agent navi- gating around an obstacle to its goal object across eight familiarization trials (a). At test, the rational agent ei- ther follows efficient path (b) inefficient path (c).
Can an AI system bind specific preferences for goal objects specific agents?
Developmental Background. Infants are capable of attributing specific object preferences specific agents (Repacholi and Gopnik, 1997; Kuhlmeier et al., 2003; Buresh and Woodward, 2007; Henderson and Woodward, 2012). For example, while 9- and 13-month-old infants looked longer at test when an actor reached for a toy that they did not prefer during a habituation phase, infants showed no expectations when the habituation and test trials featured different actors (Buresh and Woodward, 2007). When shown one agent repeatedly moving to the same object, do machines expect that that object is preferred by that specific agent?
Familiarization Trials. The familiarization shows an agent consistently choosing one object over the other, but objects appear widely varying locations in the grid world.
Test Trials. The test includes four possible outcomes: the same agent moves to the preferred object (expected); new agent moves to the object preferred by the first agent (no expectation) (appendix, Figure 12); the same agent moves to the nonpreferred object (unexpected); or the new agent moves to the nonpreferred object (no expectation) (appendix, Figure 13). The model is successful if it has the same relative expectations as listed above; that is, weak or no expectations about the preferences of the new agent compared to the familiar agent.
2.3 Can an AI system understand that physical obstacles might restrict agents' actions, and does it predict that an agent might approach nonpreferred object when the preferred one inaccessible?
Developmental Background. Infants understand the principle of solidity (e.g., that solid objects cannot pass through one another). and they apply this principle to both inanimate entities (Baillargeon, 1987; Baillargeon
al., 1992; Spelke et al., 1992) and animate entities, such as human hands (Saxe et al., 2006; Luo et al., 2009). Infants' expectations about the objects agents might approach are also informed by object accessibility. Scott and Baillargeon (2013) demonstrate, for example, that 16-month-old infants expected an agent, facing two identical objects, to reach for the one in the container without lid versus the one in the container with lid. When shown an agent repeatedly moving to the same object, do machines recognize that the agent's access to that object might change, and do they predict that an agent might then approach a nonpreferred object?

---

Familiarizatiion Trials. The familiarization shows an agent consistently choosing one ob- ject over the other, as above, and objects appear widely varying locations the grid world (Figure 4).
Pomiliarisatinn trild))
(b) Expected
Unexpected
HT
STI
PT
Test Trials. The test presents new object locations. In the no-expectation outcome, the preferred object is now blocked on all sides by fixed, black barriers, and the agent moves to the nonpreferred object. In the unexpected out- come, both of the objects remain accessible, and the agent moves to the nonpreferred object (Fig- ure 4). The model is successful if it has the same relative expectations; that is, weak or no expectation about the agent's s moving to the nonpreferred object only when the preferred object is inaccessible.
Figure 4: Can machines understand that obstacles re- strict actions? The familiarization trials present agent navigating to their preferred object varied locations (a). At test, either the preferred object is inaccessible and the agent goes to the nonpreferred object (b) or the preferred object is accessible and the agent goes to the nonpreferred object (c).
2.4 Can Al system represent agent's sequence of actions as instrumental, directed towards a higher-order goal object?
Developmental Background. Infants represent an agent's sequence of actions as instrumental to achieving higher-order goal (Carpenter al., 2005; Elsner et al., 2007; Hernik and Csibra, 2015; Gerson al., 2015; Saxe al., 2007; Sommerville and Woodward, 2005; Woodward and Sommerville, 2000). For example, Som- merville and Woodward (2005) showed that 12- month-old infants understand an actor's pulling a cloth as means to getting the otherwise out- of-reach object placed on it. When shown an agent repeatedly taking the same action to effect a change in the environment that enables them to move towards an object, do machines expect that that object the goal, as opposed to the sequence of actions?
Familiarization trials)
Test: Expected
Test: Unexpected
(a) No harriers
Familiarization (8 trials)
Test: Expected
Test: Unexpected
(b) Inconsequential barriers
Famillarization trials)
Expected
Unexpected
Familiarization Trials. The familiarization in- cludes five main elements: an agent; goal ob- ject; key; lock; and a green removable bar- rier (see Figure 5). The green barrier initially restricts the agent's access the object. And so, the agent removes the barrier by collecting and inserting the key into the lock. The agent then moves to the object.
(c) Blocking
Figure 5: Can machines recognize instrumental actions towards higher-order goals? BIB's 's three types of test trials evaluate machines' understanding of instrumen- tal actions. The agent's 's goal is initially inaccessible (blocked by green removable barrier). During familiar- ization (left), the agent removes the barrier by retrieving the key (triangle) and inserting into the lock. At test, the agent's 's moving directly to the goal is expected when the green barrier is absent (a) or not blocking the goal object (b,c); its moving to the key in those cases is unex- pected.
Test Trials. The test phase presents three differ- ent scenarios for total of six different outcomes, In the scenario with no green barrier: the agent moves directly to the object (expected); or to the key (unexpected) (Figure 5a). In the scenario with an inconsequential green barrier: the agent moves directly to the object (expected); or the key (unexpected) (Figure 5b). In the scenario with variability in the presence/absence of the green barrier: the barrier blocks the agent's access to object, and the agent moves key (expected); or, the barrier does not block the object and the agent moves the key (unexpected). The model is successful it expects the agent to go to the key only when the green removable barrier is blocking the goal object (Figure 5c).
5

---

2.5 Can an Al system predict that rational agent will move efficiently towards a goal object?
Developmental Background. Infants expect agents to move efficiently towards their goals (Gergely et al., 1995; Gergely and Csibra, 1997, 2003; Liu al., 2017, 2019; Colomer et al., 2020). In a seminal study by Gergely et al. (1995), for example, 12-month-old infants repeatedly saw small circle jumping over an obstacle to get to a big circle (see Figure 3 left). At test, the obstacle was removed, and the small circle either took the same, now inefficient, path to get to the big circle or took the straight, efficient path. Infants were surprised when the agent took the familiar, inefficient path. These findings have been replicated by instantiating the agent and object in different ways (as, e.g., humans, geometric shapes, or puppets) and by using different kinds of presentations (e.g., prerecorded or live) (Colomer et al., 2020; Phillips and Wellman, 2005; Sodian et al., 2004; Southgate et al., 2008; Liu et al., 2017). When infants see an irrational agent, i.e., one moving inefficiently to their goal from the start, however, they do not form any expectations about that agent's actions test (Gergely et al., I 1995; Liu and Spelke, 2017). When shown a rational agent repeatedly taking an efficient path around an obstacles to its goal object, do machines expect that that agent will continue to take efficient paths, as opposed to similar-looking paths, relative to the obstacles in the environment?
Familiarization Trials The familiarization includes two different scenarios: a rational agent consis- tently moves along an efficient path to its goal object around a fixed black barrier in the gird world (Figure 3a); or, an irrational agent moves along these same paths the rational agent, but there is no barrier in the way (appendix, Figure 14b).
Test Trials. The test includes two possible scenarios. One scenario shows only the rational agent, and it presents one of the familiarization trials but with the barrier between the agent and the goal object removed or changed in position (such that curved path is still required). The agent either moves along an efficient path to its goal (expected) or the agent moves along one of two unexpected paths, either the exact same, but now inefficient, path that it had during familiarization (path control, Figure 3) or along a path that is inefficient but takes the same amount of time as the efficient path (in this latter case, the goal object starts off closer to the agent, appendix Figure 11). The second scenario shows either the rational or irrational agent taking an inefficient path towards its goal. This outcome should be unexpected in the case of the rational agent, but should yield expectation in the case of the irrational agent (appendix, Figure 14). The model is successful if expects only a rational agent modify its path based the location of barriers and move efficiently to its goal.
3 Background Training
While infants in the lab make meaningful in- ferences about novel stimuli and environments with only a brief familiarization phase, BIB in- cludes tens of thousands of background episodes as a generous stand-in for this type of lab fa- miliarization. Models should therefore not be surprised merely by BIB's s elements and dynam- ies. Just as infants may already have knowledge about agents, objects, and places prior to com- ing to the lab, moreover, we do not intend to limit models to just BIB's background training prior to being tested. Although learning-centric approaches will learn something about agents if trained on the background set, either supple- mental pretraining or additional prior knowledge can be enriched by the background training for a model to approach the benchmark success- fully. The episodes in the background training are structured similarly to those in the evalua- tion, although the familiarization and test trials are drawn from the same distribution for the background training only. Similar to IntPhys (Riochet et al., 2018) and ADEPT (Smith et al.,
(x) Single object
(b) navigation Proference
(c) Agent-blocked Instramental acrion
E
(d) Single abject Multtlle--genn
Figure 6: The four tasks from the background train- ing set, including the Single-Object Task (a), the No- Navigation Preference Task (b), the Agent-Blocked Instrumental Action Task (c), and the Single-Object Multiple-Agent Task (d). For tasks (a)-(c), only the test trials are shown here. For task (d), the agent switches during faniliarization and continues during test. An ex- ample of two agents in same episode are shown here.

---

2019), we only provide expected outcomes at test for the background training. There are four training tasks:
Single-Object Task. The agent navigates to an object at some varied location in the environment (Figure 6a). This task is different from the evaluation tasks, where there are two objects or the arrangement of the barriers in the environment changes. With this training, models can learn how agents start and end trials, how agents move, and how barriers influence agents' motion. We provide 10,000 episodes of this type.
No-Navigation Preference Task. Two objects are located very close to the agent's initial starting location but at varied locations, and the agent approaches one object consistently across trials (Figure 6b). The task allows the model to learn that agents display consistent preference- basee behavior. Critically, the navigation in these trials is trivial compared to the evaluation trials, so navigation to a preferred goal object is not trained. We provide 10,000 episodes of this type.
Single-Object Multiple-Agent Task. One object located very close to the agent's initial starting location but at varied locations (Figure 6d). At some point during the episode, a new agent takes the initial agent's 's place (for example, the initial agent could be replaced at the fourth trial, and all subsequent trials, including the test trial, would have the new agent). The task allows the model to learn that multiple agents can appear across trials. This task differs from the evaluation task, where there are two objects and new agent appears only in the test trials. We provide 4,000 episodes of this type.
ggent-Blocked Blocked Instrumental Action Task. The trial starts with the agent confined to small region of the grid world, blocked by a removable green barrier (Figure 6c). The agent collects the key and inserts it into the lock to make the barrier disappear. The agent then navigates to the object. This task allows the model to learn that the green barrier obstructs navigation and that inserting the key in the lock removes that barrier. These trials differ from the evaluation in that the removable barriers are around the agent instead of the object. We provide 4,000 episodes this type.
To be successful at the evaluations, models must acquire or enrich their representations of agents for flexible and systematic generalization. For example, models have combine acquired knowledge of navigation (Single-One at the evaluation testing their understanding that agents have preferences for goal objects, not goal locations (section 1..
Task) and preferences (No-Navigation, Preference Task) to be successful
4 Baseline Models
When being evaluated on BIB, a model cannot actively sample from the environment; it can only use the samples provided in the episodes themselves. We therefore did not test baseline models using traditional approaches in imitation learning (II.,, inverse RI. (IRL), and RL (Ng et al., 2000; Abbeel and Ng. 2004; Ziebart et al., 2008) because they require substantial privileged information, such as access to the environment to actively sample trajectories using the modelled policy, and, in the case of the RL algorithms, an observable reward. Moreover, these approaches often model one agent at a time, and BIB requires the same model to infer the behavior of different agents across different episodes (although recent approaches in deep RL and IRL try to mitigate this latter issue with work in meta-RL and meta-IRL (Xu et al., 2019; Yu et al., 2019; Rakelly et al., 2019) that allows for similar cross-episode adaptation). Although this feature of BIB makes it less suitable for testing RL models, it is essential to BIB's 's design because it reflects infants' reasoning. Infants rely on little to no active interaction with particular environment to make meaningful inferences and predictions about the agents in that environment, and infants' inferences are far more abstract than their particular observed or active experience (Skerry et al., 2013; Gergely et al., 2002: Liu et al., 2019; Zmyj et al., 2009).
We thus tested three baseline models spanning different approaches including video modeling, behavior cloning (BC), and offline RL (see appendix C for full model specifications). Models were trained passively and through observation only. An episode in BIB can be broken down into nine trials with trajectories, (Ti J where 7, E [1, 8] are the familiarization trials and Ty is the test trial. Each trajectory тę consists of series of state (frames from the video) and action pairs (sij. 4:5) ) The action space of the agents in BIB is of size eight; agents can move along the two axes or along the diagonals. Additionally, the transitions are deterministic.
7

---

Inpai
Mudel Pradiction
Tanget
(a) The model predicts that the brown agent will to the green object gray object, its preferred gual object during familiirizatiinn
Inpui Prame
Target
F
FAMALOATITTOON TRIALS
LSTM
ESTM
LSTM
(b) The model predicts that blue agent will go the inaccessible eyan object instead of an accessible object.
DIMMACTENGTIC
ITECOITT TON
Inpur Frame
Model Prediction
Tanget Prame
TRVALS
CHAMMCTERETIC
agent go to inaccessible orange object instead of performing the instrumental action to first collect triangular key.
model predicts
Figure 7: Architecture the video baseline model inspired by Rabinowitz et al. (2018). An agent- charrttccacttiiitti em- bedding inferred from the familiarization trials using recurrent This embedding, with frame from the test trial, used to predict next frame the video using a U-Net (Ronneberger et al., 2015).
Figure 8: The most surprising frame (the frame with the highest prediction error) from the test trial for the video model taken from the evaluation tasks. Examples of failed expectations shown here.
Our baseline models either predict the next frame the video (see Figure for architecture) or the actions taken by the agent (see appendix C.2). To encode the context in the form of the familiarization trials, we use sequence of frames (for the video model) and frame-action pairs (for the BC models). In terms of the architecture, the baseline models take inspiration from state-of tfate-of-the-art neural- network-baser approach to encode the characteristic of an agent: the theory of mind net (ToMnet model in Rabinowitz al. (2018). We encode the familiarization trials context using either bidirectional L.TT or an MIP. In addition to video modeling and BC, we also try an offline-RL. baseline (Siegel al., 2020). Offline-RL. algorithms (Levine et al., 2020) policy from demonstrations by using privileged information in the form of rewards received by the agent. We engineer reward function based on the distance of the agent from the goal train the RL policy (see appendix C.3).
designed learn
During evaluation, the "expectedness" of test trial, in the context of the previous familiarization trials, was inferred by model's error on the most 'unexpected" step (i.e., the step with the highest prediction error). The most 'unexpected' step was chosen for comparison alternatives (like the mean expectedness of steps) consistently resulted lower VOE scores. For each evaluation episode, first calculated the model's relative accuracy, i.e., whether the model found the expected video in each pair more expected than the unexpected video (chance is 50%).
Results
The models were trained on 80% the background training episodes (training set), and the rest of the episodes were used for validation (validation set). A comparison of the MSE loss (on pixels for the video model and in the action space for the BC and RL models) on the training and validation sets indicated that the models had learned the training tasks successfully (see appendix C).
The results of our baselines are presented Table The offline model performs similarly to the BC approach (see appendix C.3 for details), so we do not consider the offline RI model further (offline RL is known to provide little improvement over BC when the demonstrations are not noisy (Siegel et al., 2020)). Comparing the performance of BC on BIB and AGENT (Shu et al.,

---

BIB AGENCY TASK
BC-MLP
BC-RNN
VIDEO-RNN
PREFERENCE
26.3
48.3
47.6
MULTI-AGENT
48.7
48.2
50.3
INACCESSIBLE GOAL
76.9
81.6
74
EFFICIENCY: PATH CONTROL
94.0
92.8
99.2
EFFICIENCY: TIME CONTROL
99.1
99.1
99.9
EFFICIENCY: IRRATIONAL AGENT
73.8
56.5
50.1
EFFICIENT ACTION AVERAGE
88.8
82.5
83.1
INSTRUMENTAL: No BARRIER
98.8
98.8
99.7
INSTRUMENTAL: INCONSEQUENTIAL BARRIER
55.2
78.2
77.0
INSTRUMENTAL: BLOCKING BARRIER
47.1
56.8
62.9
INSTRUMENTAL ACTION AVERAGE
67.0
77.9
79.9
Table 1: Performance of the baseline modcls BIB, The scores quantify pairwise VOE judgements.
2021), we see that BC performs worse on BIB. BC and offline-RI. are known to perform poorly on out-of-distribun and systematically different scenarios from training to test (Siegel et al., 2020). Because systematic generalization is primary feature of BIB but not AGENT, this is likely why BC performs worse on BIB.
The models we tested with RNN (Video-RNN; BC-RNN) perform at chance on the Preference Task (see Figure 8a for predictions made by the video model); they tend to predict that an agent will go to the closer object (this prediction is made in about 70% of trials). The models thus ignore the agent's 's preference, established during familiarization. This finding is especially striking relative to the models' success on the No-Navigation, Preference Task from the background training and could result from differences in the distance at which the objects are placed in the scene. In the background training, the objects are close to the agent, yielding short trial lengths during familiarization. The characteristic encoder RNN might find it difficult to generalize to the evaluation tasks' longer sequences. The BC-MLP model confused by how the object locations correlate with their identity, encoding agent's preference for goal location instead of goal object. This too is surprising, as the background training provides evidence that agents prefer object identities, not specific locations. None of these models, result, succeed in recognizing that agents have preferences and goals for specific objects.
The models also fail on the Multi-Agent Task, again tending to predict that an agent will go to the closer object regardless of any established preferences. Consistent with this failure, the models also fail to map specific preferences to specific agents. The models do slightly better than chance on the Inaccessible Goal Task. As seen Figure 8b, video model still, nevertheless, frequently predicts that the agent will go to the inaccessible goal. The models are proficient at finding the shortest path to the goal in the Efficiency Task (appendix Figure 16a), leading to high accuracy on both sub-evaluations that test for efficient action: Path Control and Time Control (Table 1). However, the two models with an RNN fail to modulate their predictions based on whether the agent was rational or irrational during familiarization (Table 1). The BC-MLP has a weak expectation of rationality from an irrational agent, scoring slightly higher that the two RNN models on this task. Finally, the models perform above chance the Instrumental Action Task, but performance (Table 1) indicate that they rely on the simple heuristic of directly going to the goal object rather than understanding the nature of the instrumental action (Figure 8c). This heuristic leads to higher scores on sub-evaluations with no barrier and an inconsequential barrier (Table 1) but lower scores on the sub-evaluation with blocking barrier. This poor performance may be due to the difference in the relations between the agent and barrier in the background training (where the agent is confined; Figure 6c) and evaluation (where the object is confined; Figure 5).
the sub-evaluations
6 General Discussion
We introduced the Baby Intuitions Benchmark (BIB), which tests machines on their ability to reason about the underlying intentionality of other agents by only observing their actions. BIB directly inspired by the abstract reasoning about agents that emerges early in human development, as revealed by behavioral studies with infants. BIB s adoption of the VOE paradigm, morcover, means both that its results can be interpreted in terms of human performance and that low-level heuristics can be

---

directly evaluated. While baseline, deep-learning models successfully generalize to BIB's 's training tasks, they fail systematically generalize to the evaluation tasks even though the models incorporate seeyy--f--inn-iispiipe
architectures (Rabinowitz et al., 2018).
What kinds of models might succeed BIB? Shu et al. (2021) proposes Bayesian inverse planning model for success on AGENT's agency-reasoning tasks. Extending this structured Bayesian approach BIB is not straightforward. however, as requires features that are not currently provided. Recent approaches in deep imitation learning and inverse RL, especially work in meta-IL and meta-IRL (Xu al., 2019; Yu et al., 2019) that allows for test-time adaptation, also show promise and could ultimately lead to more human-like reasoning about other agents. Extending these approaches to BIB nevertheless also non-trivial since they require active sampling of the environment, which is not something that BIB allows. We hope that BIB will catalyze future research in these directions.
While the comparison between human and machine performance will he bolstered by future hehavioral studies with infants, the results from the baseline models already shed new light on critical differences between human and artificial intelligence (also see appendix D). For example, across tasks, the baseline models tended to predict that an agent would move to the closer of two objects, regardless of the agent's 's previously demonstrated preference for only one of the objects. And, while models succeeded in predicting that an agent would move efficiently to an object, they over-generalized that efficiency principle to include agents who had previously demonstrated inefficient, irrational actions. For infants, object- based preferences and efficient, goal-directed action instead serve to enrich their understanding of the intentions of others. For example, when infants observe an agent going farther to particular object or exerting more effort to reach it, they can attribute both a preference for that object to that agent and a value to that preference (Liu et al., 2017).
BIB also raises new questions about the foundations of common--sens. reasoning about agents for both humans and machines alike. For example, the "extended familiarization" needed for training Al models (i.e., the background training) potentially reveals a striking difference between how BIB might challenge minds versus machines. While both infants and AI systems may have built-in knowledge and/or experience prior to participating in BIB, infants likely need only eight, as opposed to thousands, of videos of shapes moving around grid worlds to successfully apply their reasoning about agents to new, test events presented in that medium. Nevertheless, it remains unknown whether infants might also benefit from some kind of background training meant for machines; such designs have never been tested with infants. Relatedly, BIB challenges the generalizability of early emerging, human common-sense reasoning about agents. How well do humans recognize simple shapes with simple movements and minimal cues to animacy (e.g., no eyes/gaze direction, no distinctive sounds, and no emotional expressions) as agents with intentionality? Do navigation- versus reaching-content deferentially shift humans' attention to locations versus objects as candidates for an agent's 's goal? How does a comprehensive set of reeccy-reasonig abilities relate to one another within the same individual? Most of the existing infant literature on which BIB is based presents infants with richer cues to animacy (in the form of live-action or animated displays from frontal or three-quarters points of view), events with no minimal navigation to establish preferences for goal objects, and individual tests of one competency or another, outside of a unified framework.
The origins and development of humans' intuitive understanding of agents and their intentional actions have been studied extensively in developmental cognitive science. The representations and computations underlying such understanding. however, are not yet understood. BIB serves test for computational models with different priors and learning-based approaches to achieve common-sense reasoning about agents like human infants. A computational description of how we reason about agents could ultimately help us build machines that better understand us and that we better understand.
Finally, BIB serves as key step in bridging machines' impoverished understanding of others' mental states with humans' rich one. To achieve a human-like theory of mind, a model would not only have to understand intentionality, which is tested in BIB, it would also have to understand the perceptions and beliefs of other agents. A combined, comprehensive understanding may underlie human theory of mind and lead to, for example, success in false belief task, in which humans have expectations about where an agent would search for a goal object based on where that agent last saw it (Baron-Cohen et al., 1985; Spelke, 2016). A benchmark that focuses on reasoning about agents' phenomenological and epistemic states is thus a natural extension of BIB and could further advance our understanding of both human and artificial intelligence.
10

---

Acknowledgments and Disclosure of Funding
This worked was supported by the DARPA Machine Common Sense program (HRO01 1900000) We thank Victoria Romero, Koleen McKrink, David Moore, Lisa Oakes, and Clark Dorman for their generous feedback. We are also grateful Thomas Schellenberg. Dean Wetherby, and Brian Pippin for their development effort in porting the benchmark to 3D. Finally, we thank Brian Reilly for coming up with the name of the benchmark and finding the perfect acronym for our work.
References
Abbeel, P. and Ng, A. Y. (2004). Apprenticeship learning via inverse reinforcement learning. In Proceedings of the 21st International Conference Machine Learning.
Abdolmaleki, A., Springenberg. J. T., Tassa, Y., Munos, R., Heess, N., and Riedmiller, M. (2018). Maximum posterion policy optimisation. In International Conference Learning Representa- tions.
Albrecht, S. V. and Stone, P. (2018). Autonomous agents modelling other agents: A comprehensive survey and open problems. Artificial Intelligence, 258:66-95.
Baillargeon, R. (1987). Object permanence in 31/2-and 41/2-month-old infants. Developmental Psychology, 23(5):655.
Baillargeon, R., Needham, A., and De J. (1992). The development of young infants' intuitions about support. Early Development and Parenting, 1(2):69-78.
Baillargeon, R., Scott, R. M., and Bian, L. (2016). Psychological reasoning in infancy, Annual Review of Psychology, 67: 67:159-186.
Baillargeon, R., Spelke, E. S., and Wasserman, S. (1985). Object permanence in five-month-old infants. Cognition, 2013):191-208
Baker, C., Saxe, R., and Tenenbaum, J. (2011). Bayesian theory of mind: Modeling joint belief-desire attribution. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 33.
Baker, C. L., Jara--ttinger J., Saxe, R., and Tenenbaum, J. B. (2017). Rational quantitative attribution of beliefs, desires and percepts in human mentalizing. Nature Human Behaviour, 1(4):1-10.
Baker, C. L., Saxe, R., and Tenenbaum, B. (2009). Action understanding as inverse planning. Cognition, 113(3):329-34,,
Banaji, M. and Gelman,
(2013). Navigating the social world: What infants, children, and other species teach us. Oxford University Press.
Baron-Cohen, S., Leslie, A. M., and Frith, U. (1985). Does the autistic child have "theory of mind"? Cognition, 21 21(1)37746.
Buresh, S. and Woodward, A. L. (2007). Infants track action goals within and across agents. Cognition, 104(2):287-314
Carpenter, M., Call, J., and Tomasello, M. (2005). Twelve-and 18-month-olds copy actions in terms of goals. Developmental Science, F33FF13-
Colomer, M., Bas, J., and Sebastian-Galles, N. (2020). Efficiency as a principle for social preferences in infancy. Journal of Experimental Child Psychology, 194:104823
Elsner. B., Hauf, P., and Aschersleben, G. (2007). Imitating step by step: A detailed analysis of 9-to 15-month-olds' reproduction of a three-step action sequence. Infant Behavior and Development, 30(2):325-335.
Gergely, G., Bekkering, H., and Király, I. (2002). Rational imitation in preverbal infants. Nature, 415(6873):755-755

---

Gergely, G. and Csibra, G. (1997). Teleological reasoning in infancy: The infant's 's naive theory of rational action: A reply to premack and premack. Cognition, 63(2):227-233.
Gergely, G. and Csibra, G. (2003). Teleological reasoning in infancy: The navee theory of rational action. Trends in Cognitive Sciences, 7(7):287 292..
Gergely, G., Nádasdy, Z., Csibra, G., and Bíró, S. (1995). Taking the intentional stance at 12 months of age. Cognition, 195333.6.
Gerson, S. A., Mahajan, N., Sommerville, A., Matz, L., and Woodward, A. L. (2015). Shifting goals: Effects of active and observational experience on infants' understanding of higher order goals. Frontiers in Psychology, 6:310.
Heider, F. and Simmel, M. (1944). An experimental study of apparent behavior. The American Journal of Psychology, 57(2):243-22
Henderson, A. M. and Woodward, A. L. (2012). Nine-month-old infants generalize object labels, but not object preferences across individuals. Developmental Science, 15(5):641-652
Hernik, M. and Csibra, G. (2015). Infants learn enduring functions of novel tools from action demonstrations. Journal of Experimental Child Psychology, 130:176-192.
Ho, J. and Ermon, S. (2016). Generative adversarial imitation learning. Advances in neural informa- tion processing systems, 29:4565-4573.
Jara-Fttinger, J. (2019). Theory of mind inverse reinforcement learning. Current Opinion in Behavioral Sciences, 29: 105-110.
Kolve, E., Mottaghi, R., Han, W., VanderBilt, E., Weihs, L.,, Herrasti, A., Gordon, D., Zhu, Y., Gupta, A., and Farhadi, A. (2019). Ai2-thor: An interactive 3d environment for visual ai.
Kuhlmeier, V., Wynn, K., and Bloom, P. (2003). Attribution of dispositional states by 12-month-olds. Psychological Science, 14(5):402-408
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., and Gershman, S. (2017). Building machines that learn and think like people. Behavioral and Brain Sciences, 40.
Levine, S., Kumar, A., Tucker, G., and Fu, J. (2020). Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643
Liu, S., Brooks, N. B., and Spelke, E. S. (2019). Origins of the concepts cause, cost, and goal in prereaching infants. Proceedings of the National Academy of Sciences, 1 114767717777--775
Liu, S. and Spelke, E. S. (2017). Six-month-old infants expect agents to minimize the cost of their actions. Cognition, 160:35-42.
Liu, S., Ullman, T. D., Tenenbaum. J. B., and Spelke, S. (2017). Ten-month infants infer the value of goals from the costs of actions. Science, 183866666:108811000
Luo, Y. (2011). mhree-month-old infants attribute goals to non-human agent. Developmental Science, 14(2):553 460.
Luo, Y. and Baillargeon, R. (2005). Can a self-propelled box have a goal? psychological reasoning in 5-month-old infants. Psychological Science, 16(8):601-608.
Luo, Y., Kaufman, L., and Baillargeon, R. (2009). Young infants' reasoning about physical events involving inert and self- propeleddl objects. Cognitive Psychology, 58(4):441-48)
Ng, A. Y., Russell, S. J.. et al. (2000). Algorithms for inverse reinforcement learning. In Proceedings of the 17th International Conference on Machine Learning, volume 1, page 2.
Oakes, L. M. (2010). Using habituation of looking time to assess mental processes in infancy. Journal of Cognition and Development, 133):255-268.
Phillips, A. T. and Wellman, H. M. (2005), Infants' understanding of object-directed action. Cognition, 1875):137-155
12

---

Premack, D. and Woodruff, G. (1978). Does the chimpanzee have theory of mind? Behavioral and Brain Sciences, 1(4):515-526
Rabinowitz, N., Perbet, F., Song, F., Zhang, C., Eslami, S. M. A., and Botvinick, M. (2018). Machine theory of mind. In Dy, and Krause, A., editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 4218 Stockholmsmässan, Stockholm Sweden. PMLR.
Raileamu, R., Denton, E., Szlam, A., and Fergus. R. (2018). Modeling others using oneself in multi- agent reinforcement learning. In International conference machine learning, pages 4257-4266 PMLR.
Rakelly, K., Zhou, A., Finn, C., Levine, S., and Quillen, D. (2019). Efficient off-policy meta- reinforcement learning via probabilistic context variables. In International Conference on Machine Learning, pages 5331-5340. PMLR.
Repacholi, B. M. and Gopnik, (1997). Early reasoning about desires: evidence from 14-and 18-month-olds. Developmental Psychology, 33(1):12.
Riochet, R., Castro, M. Y., Bernard, M., Lerer, A.,, Fergus, R., Izard, V., and Dupoux, E. (2018). Int- phys: A framework and benchmark for visual intuitive physics reasoning. CoRR, Abs//803.07616
Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer- Assisted Intervention, pages 234 241.. Springer.
Saxe, R., Tzelnic, T., and Carey, S. (2006). Five-month-o infants know humans are solid, like inanimate objects. Cognition, 101(1):B1-BB)
Saxe, R., Tzelnic, T., and Carey, S. (2007). Knowing who dunnit: Infants identify the causal agent in an unseen causal interaction. Developmental Psychology, 43(1):149,
Scott, R. M. and Baillargeon, R. (2013). Do infants really expect agents to act efficiently? a critical test of the rationality principle. Psychological Science, 24(4):466 4444
Shimizu, Y. A. and Johnson, S. C. (2004). Infants' attribution of goal to a morphologically unfamiliar agent. Developmental Science, 7(4):425-430.
Shu. T., Bhandwaldar, A., Gan, C., Smith, K., Liu, S., Gutfreund, D., Spelke, E., Tenenbaum, J. B., and Ullman, T. D. (2021). AGENT: Benchmark for Core Psychological Reasoning. arXiv preprint arXiv:2 12321.
Siegel, N., Springenberg. J. T., Berkenkamp, F., Abdolmaleki, A., Neunert, M., Lampe, T., Hafner, R., Heess, N., and Riedmiller, M. (2020). Keep doing what worked: Behavior modelling priors for offline reinforcement learning. International Conference Learning Representations.
Skerry, A. E., Carey, S. E., and Spelke, E. S. (2013). First-person action experience reveals sensitivity to action efficiency in prereaching infants. Proceedings of the National Academy of Sciences, 110(46)117228-188)
Smith, K., Mei, L.,, Yao, S., Wu, J., Spelke, E.,, Tenenbaum, J., and Ullman, T. (2019). Modeling expectation violation in intuitive physics with coarse probabilistic object representations. In Wallach, H., Larochelle, H., Beygelzimer, A., d' d'Alché-Buc F., Fox, E., and Garnett, R., editors, Advances in Neural Information Processing Systems 32, pages 8985-8995. Curran Associates, Inc.
Sodian, B., Schoeppner, B., and Metz, U. (2004). Do infants apply the principle of rational action to human agents? Infant Behavior and Development, 27(1):31-41
Sommerville, J. A. and Woodward, A. L. (2005). Pulling out the intentional structure of action: the relation between action processing and action production in infancy. Cognition, 95(1):1-30.
Song, H.-j.. Baillargeon, R., and Fisher, C. (2005). Can infants attribute to an agent a disposition to perform particular action? Cognition, 98(2):B45-B55
13

---

Southgate, V., Johnson, M., and Csibra, G. (2008), Infants attribute goals to biomechanically impossible actions. Cognition, 107(3):059--069.
Spelke, E. S. (2016). Core knowledge and conceptual change, chapter 16, pages 279-300. Oxford University Press.
Spelke, E. S., Breinlinger, K., Macomber, J., and Jacobson, K. (1992). Origins of knowledge. Psychological Review, (9(4))605
Stooke, A., Lee, K., Abbeel, P., and Laskin, M. (2021). Decoupling representation learning from reinforcement learning. International Conference on Machine Learning.
Turk-Browwn B., Scholl, B. J., and Chun, M. M. (2008). Babies and brains: habituation in infant
cognition and functional neuroimaging, Frontiers in Human Neuroscience, 2:16.
Ullman, T., Baker, C., Macindoe, O., Evans, O., Goodman, N., and Tenenbaum, J. B. (2009), Help
hinder: Bayesian models of social goal inference. In Advances Neural Information Processing Systems, pages 1874-1882.
Woodward, A. L. (1998). Infants selectively encode the goal object of an actor's reach. Cognition, 69(1):1-34.
Woodward, A. L. (1999). Infants' ability to distinguish between purposeful and non-purposeful behaviors. Infant Behavior and Development, 22(2):145-160.
Woodward, A. L. and Sommerville, A. (2000). Twelve-month-old infants interpret action in context. Psychological Science, 11(1):73-77.
Xu, K., Ratner, E., Dragan, A., Levine, S., and Finn, C. (2019). Learning prior over intent via meta-inverse reinforcement learning. International Conference on Machine Learning, pages PMLR.
Yu, L.,, Yu, T., Finn, C., and Ermon, S. (2019). Meta- inverse reinforcement learning with probabilistic context variables. In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., and Garnett, R., editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.
Ziebart, B. D., Maas, A. L., Bagnell, J. A., and Dey. A. K. (2008). Maximum entropy inverse reinforcement learning. Aaai, volume 8, pages 1433-1438. Chicago, IL. USA.
Zmyj. N., Daum, M. M., and Aschersleben, (2009). The development of rational imitation in and 12-month-old infants. Infancy, 14(1):131-141
14


Reviews:
[{"review_id": "Wzr1c-TSRP1", "reviewer": "Reviewer_uke7", "summary": "This paper presents the Baby Intuitions Benchmark (BIB) to test deep learning methods abilities’ to reason about how other agents will behave. BIB is inspired by human infants’ expectations about other agents, as clearly laid out in the paper. The authors include several different capabilities/tasks within the benchmark along with their developmental background, testing 3 different models across 5 different tasks. These models only perform well in the simplest of tasks, suggesting that the benchmark tests capabilities that are not trivial to acquire with current methods.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "To summarize my review, I found the paper compelling as a pointer to how rich human social inferences can be, and just how lacking a variety of ML baselines are in this respect. My most critical technical comments relate to including a model that relies on bayesian inference, as motivation for how well models could perform if they had the right kinds of structure in them (as in AGENT [1]). While I am not sure how much this benchmark will be picked up by the general machine learning community, it does point out specific limitations with current methods for social inference, and could therefore be useful especially for researchers in human computer interaction, human robot interaction, and multi-agent ML researchers generally. \n\nHowever, I am choosing a low rating because this is not the appropriate track for this paper. This paper does not make a meaningful contribution for the main NeurIPS track, in that it does not show new results either for cognitive science/neuroscience, or provide new algorithms for machine learning methods. It instead tests a variety of existing methods on a new benchmark. It should absolutely be submitted to the “Datasets and Benchmarks” track. If it is possible to move the paper to this other track, my rating would jump to 7 for that track. This alternative track has a deadline for submission of August 27, please see https://neurips.cc/Conferences/2021/CallForDatasetsBenchmarks for details.\n\nFor context, I think that for a benchmark to be accepted to the main track, it needs to either (1) provide access to data that would otherwise be difficult for individual researchers to collect, like neural recordings, human annotations, or human experimental data, or (2) clearly address an important ML problem, and provide clear reasons for why the benchmark will be very widely taken up within the ML community. While I like the BIB benchmark, it does not fit into (1), and I am not convinced that it clearly falls into category (2) either (due to similarities to existing benchmarks, as well as BIB’s somewhat bespoke passive setting where agents cannot interact with the environment), which is why I strongly recommend considering the datasets and benchmarks track. \n\n### Excellent paper layout - very clear writing\n\nThe section headings - stating the question of the specific sub-task - and then giving the developmental background and the specifics of the familiarization and test trials, was very clear. My only additional suggestion would be to include a small wrapped figure in each of these sections (like Figures 4, 5, and 6) to help a reader visualize what the task looks like as well. \n\n### Reasonable ML baselines, although could include bayesian inference agent\n\nThe baselines included by the authors, an offline RL, behavioral cloning, and video RNN models, seemed reasonable for the presented task to cover a variety of approaches that ML researchers might try first. The authors explained very well why inverse reinforcement learning approaches are not suitable for this benchmark, precisely because agents do not have access to the underlying environment to attempt to create their own plans. \n\nMy only suggestion would be to include an “aspirational” baseline as in AGENT [1], that uses significantly more structure (in the form of objects and agents), but as a result is able to solve the presented tasks. I think this could help convince readers that the benchmark is interesting, as it pushes on building these more structured representations in order to enable success. Without something like this, researchers might think the benchmark is just too difficult, and not try it. \n\n### Benchmark's passive nature may prevent widespread adoption\n\nThe benchmark's passive nature is a limitation in that most real setups with human-computer interaction, for example, do give agents access to the environment, and this helps dramatically with tasks like goal inference and inverse reinforcement learning. As a result, I think this benchmark as proposed will be slightly less relevant for the wider swath of the NeurIPS community, although it is well motivated in the context of developmental cognitive science.\n\n### Behavioral validation incomplete - do infants really perform at 100% accuracy for these problems?\n\nA lingering question I had throughout looking at the paper was: how well would infants really perform on these tasks? As I understand the developmental literature for these kinds of tasks, it is generally not the case that 100% of infants choose the correct stimulus 100% of the time. Can the authors comment on this? Is there a success percentage less than 100 that would be sufficient to say that a machine learning model has performed at the same level as an infant for some of these different tasks?\n\n### Questions about benchmark size\n\nThe authors use a breakdown of 10,000, 10,000, 4000, and 4000 tasks in the “background” training. Can the authors please clarify why they used this particular breakdown?\n\n[1] Shu et al, ICML 2021. AGENT: A Benchmark for Core Psychological Reasoning", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I see no obvious limitations/societal impact with the proposed benchmark.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "TOlPzppg4zj", "reviewer": "Reviewer_Yh1E", "summary": "\nThe paper starts from developmental studies on infants that showcase that infants have ways of ascribing certain types of intentionalities in what they observe, and proposes a benchmark dataset to evaluate whether modern AI/ML techniques are capable of the same type of reasoning. The paper presents a list of scenarios that demonstrate varying forms of intentionality, and also evaluates certain baseline AI/ML techniques on the benchmark scenarios to demonstrate the insufficiency of the baseline. ", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I will start with my conclusion:  \n\nWould I use the benchmark if it were available? Yes, in some form or another. At the very least as inspiration to develop new techniques within my area of interest, even if this might be different from what the authors have in mind (which seems to be deep learning over videos)\n\nIs the paper \"scientifically\" optimal? No, neither in terms of presentation/motivation, nor in terms of convincing that the problem presented would end up with machines that have a notion of intentionality.\n\nThe authors mention the existence of another similar dataset recently released (which I did not check). I would expect a more elaborate discussion (not confined to a single paragraph) of the commonalities and differences.\n\nAlthough I understand the underlying reason of why deep learning models are \"expected\" to fail the BIB tests, I am not sure whether a theory of \"object preference\" is needed for machines to do well in these tests. What is there to preclude an experimenter from biasing the deep learning in a manner that gives preferences to objects rather than location? Is there even a meaningful way to evaluate whether a deep learning architecture learns infant-like preferences, or ends up finding \"shortcuts\" to these tasks?\n\nSince not all 8 familiarization instances are shown, it is hard to know what expectations one is expected to form in some of the presented scenarios. For example in the scenario of section 2.3, what in the familiarization instances should lead one to the expectation that if the preferred object is inaccessible, then reach for the other object? Shouldn't at least one such case be in the training data? Or are the two test instances shown simultaneously, and it is a matter of relative preference between the two?\n\nI didn't quite understand the material offered. Is it several thousand (instead of 8) familiarization instances, and then several (instead of 1-2) test instances? Or is it several thousand episodes each comprising 8 familiarization instances and 1-2 test instances? In the latter case, is this really replicating the infant scenarios? I would say no, since the ML is learning also from the test instances (within the training episodes). This is clearly okay (since the evaluation episodes are not used), but it seems to change the setting from that in the infant developmental studies.\n\nIt is also unclear whether the trials (within each episode) show actual movement or just arrows indicating the movement. Figure 7 suggests the former, but up to this point in the paper the latter seems to be implied.\n\nSome of my questions above are addressed in the appendix, but perhaps not so clearly in the main text. I think there is room to clarify things in the main text as well. I would propose that one of the scenarios (e.g., preference for target object over target location) is used throughout to demonstrate in more detail all the concepts, and then the rest of the scenarios can be presented at a higher-level. Also, section 3 seems to come rather late in the narrative. Perhaps the paper presented from the point of view of ML rather than developmental psychology would offer a more compelling narrative, given the target conference?\n\nI am not sure if the 2D and 3D videos are necessary. Why not simply offer a symbolic representation (what you have in the JSON files) as the input?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Seems fine.", "review_text": "", "extra_scores": {"ethical_concerns": "Not something obvious.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3-4"}, "extra_text": {}}, {"review_id": "LsB7wRBVeSP", "reviewer": "Reviewer_dLBP", "summary": "Directly inspired by the literature on infant cognition, the authors propose a benchmark to challenge machines' social cognition with a violation of expectation paradigm.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "+ This paper is exceptionally well-motivated and tackles one of the most critical topics in developmental psychology from a machine learning perspective. It provides reasonable problem setups.\n+ The experiments and protocols are appropriately chosen. The experiments were selected from classic scenarios in developmental psychology. The VOE protocol is also basically the standard in the field.\n- Although the experiments replicate the classic studies in developmental, and the questions proposed in the paper are valid, how could the experiments and the study designs prevent the learning system to simply convert this problem into a simple classification problem, therefore may not eventually possess the abilities that the authors hope to give to the machines.\n- Of note, this paper was submitted to ICML and almost made it; I was one of the reviewers. The long meta-review commented that all reviewers find the paper interesting but lack proper baseline. In the current revision, the authors did add an additional baseline.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "See main review", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1"}, "extra_text": {}}, {"review_id": "6GgtbQYeJA-", "reviewer": "Reviewer_KaGu", "summary": "EDIT AFTER AUTHOR RESPONSE:  One concern I originally had was whether researchers would actually find this dataset useful, but the review by @Yh1E and @uke7 improved my impression of the utility of the dataset.  I am also pleased that the authors will comment more on the relation of their problem setup to classical MDP and POMDP frameworks.  Hence I am raising my score from 5 to 7.\n\nThis paper asks whether AI systems can demonstrate similar theory of mind (ToM) capabilities as human infants.  It identifies a number of key ToM capabilities that have been established in the cognitive science literature, such as learning preferences of observed agents, and predicting that agents will take the most efficient path to their goal.  The paper introduces a benchmark dataset with 2d and 3d videos with simulated agents for the purpose of testing ToM capabilities in AI systems.  Each task in the dataset consists of a familiarization phase with videos intended to allow learning of agent behavior, and then a test phase where the agent exhibits either expected or unexpected behavior.  Taking inspiration from the fact that human babies look longer at unexpected stumli, the paper introduces a \"violation of expectations (VoE)\" framework for evaluating AI systems.  Under this paradigm, an AI system is considered to demonstrate evidence of human-like learning if it has greater prediction error for video frames in the unexpected condition than in the expected condition for the testing episode.  It evaluates two recent approaches on this dataset, behavior cloning, and offline RL.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality:  High.  The paper takes a very different approach to the problem than comparable recent works in RL.  It doesn't start with any formal theory of how we might model a real-world agent (e.g. POMDP) but rather takes the cognitive science literature as a starting point.  The strong grounding in human cognitive science is a strength of this work.  The cognitive science perspective is also critical for motivating a new paradigm for evaluating artificial agents: testing whether artificial agents experience similar prediction errors to humans when seeing agents demonstrate behavior that is inconsistent with previously observed behavior.  That said, it is not clear the degree to which VOE adds to agent evaluation beyond correct prediction. If an artificial agent has high predictive accuracy, doesn't it also pass the VOE benchmark?  How does VOE add non-redundant information about the ToM capabilities of an artificial agent?\n\nQuality:  Mixed.  For a psychology venue, this would be a great paper.  But relative to the ML literature, the technical sophistication here is lacking.  The paper starts off with the cognitive science literature, which is great, but it does not go further to completely bridge to the formalisms that have been used in the artificial intelligence literature.  The technical contribution of this paper would be elevated if the paper made a serious effort to consider what human learning capabilities say about the kinds of agent models humans are likely to employ.  For instance, is there a particular type of POMDP model that would result in an agent that has stable preferences, which takes instrumental action to reach goals, and which takes the path of least effort to get there?  Another potential contribution that was not addressed by this paper is to think about how formalisms used in AI could be used to better understand infant cognition.  To what extent are infants able to learn to predict general POMDP agents?  The results also cast doubt on the correctness of the evaluation.  In the Shu et al. paper, the BC method was able to achieve above-chance prediction on the goal preference task, so why does it get below-chance performance in the BIB dataset?  How do we know that the BIB dataset has sufficient training data to enable performance comparable to human infants?\n\nClarity:  The paper is clearly written, although the VOE testing criteria could be emphasized more as it is rather novel.\n\nSignificance:  Moderate.  It seems valuable to look at human behavior to motivate benchmarks for AI, but the benchmark introduced here still seems rather limited.  Even if an artificial agent could be developed that would pass all of the tasks in the benchmark, I am not convinced that this would be conclusive evidence for the agent to have strong theory of mind.  The fact that BC and IRL perform poorly could be an important finding, but it depends on the reason for their failure, which has not been conclusively established here.\n\nOther comments:\n* Supplemental materials - some videos are mislabeled (expected switched with unexpected).  I could not play some of the videos (e.g. eval-videos-3d/efficient_action_b_expected.mp4).\n* Line 322: \"characteristics\"", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The paper does not adequately address the limitations of the benchmark, as there is no attempt to rule out alternative means that artificial agents without ToM capabilities might still be able to pass the BIB benchmarks.  The performance of benchmark methods is surprisingly bad and needs to be explained.  It is not clear that BIB dataset has enough or sufficiently varied training data to allow for generalization.\n\nThe paper could be improved by further explanation of the performance of the methods that were evaluated, addressing the adequacy of the amount and diversity of training data, and the paper should also spend some effort attempting to rule out potential alternative explanations for a method to pass the BIB benchmark.  The paper could also discuss the extent to which the VOE paradigm adds to the existing state-of-the-art in evaluating artificial agents, beyond the naturalistic argument of attempting to match human behavior.  After all, one does not have to imitate human behavior to achieve human-level performance, as human-like performance may plausibly emerge as a by-product of trying to optimize a simpler criterion.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces a new benchmark set for visual cognition tasks, inspired by looking-time experiments done with human babies. I think this work would be useful even if the only role were to highlight the questions and methodologies used by developmental psychology to student non-verbal intelligence, but care was taken to provide a broad benchmark that I think will provoke interesting ML work. The reviewers generally agree, though note a few requests for clarification and context.

(Note, it's probably the case that this paper fits better into the D&B track, but since that track is brand new this year I don't think the authors should be penalized for choosing the main track.)
Decision: Accept (Poster)