Learning an Object-Based Memory System

1 INTRODUCTION . Consider a robot operating in a household , making observations of multiple objects as it moves around over the course of days or weeks . The objects may be moved by the inhabitants , even when the robot is not observing them , and we expect the robot to be able to find any of the objects when requested . We will call this type of problem entity monitoring . It occurs in many applications , but we are particularly motivated by the robotics applications where the observations are very high dimensional , such as images or point clouds . Such systems need to perform online data association , determining which individual objects generated each observation , and state estimation , aggregating the observations of each individual object to obtain a representation that is lower variance and more complete than any individual observation . This problem can be addressed by an online recursive filtering algorithm that receives a stream of object detections as input and generates , after each input observation , a set of hypotheses corresponding to the actual objects observed by the agent . When observations are closely spaced in time and objects only briefly go out of view , the entity monitoring problem becomes the well studied problem of object tracking . In contrast , in this paper , we are interested in studying the more generalized entity monitoring problem , where a robot must associate a set of sparse and temporally separated observations of objects over the course of days or weeks into a coherent estimate of the underlying objects and associated properties in a scene ( Figure 1 ) . In such a setting , it is important that the system does not depend on continuous visual tracking , as any individual object may be seen at one time and then again significantly later . A sub-problem of generalized entity monitoring corresponds to object identification , in which we seek to consistently re-identify objects across time . However , to solve the generalized entity monitoring problem , a system must not only identify similar objects across time , but integrate the observations into an estimate of their properties that may not be directly inferrable from any single observation . A classical solution to the entity monitoring problem , developed for the tracking case but extensible to other dynamic settings , is a data association filter ( DAF ) ( the tutorial of Bar-Shalom et al . ( 2009 ) provides a good introduction ) . A Bayes-optimal solution to this problem can be formulated , but it requires representing a number of possible hypotheses that grows exponentially with the number of observations . A much more practical , though less robust , approach is a maximum likelihood DAF ( ML-DAF ) , which commits , on each step , to a maximum likelihood data association : the algorithm maintains a set of object hypotheses , one for each object ( generally starting with the empty set ) and for each observation it decides to either : ( a ) associate the observation with an existing object hypothesis and perform a Bayesian update on that hypothesis with the new data , ( b ) start a new object hypothesis based on this observation , or ( c ) discard the observation as noise . As the number of entities in the domain and the time between observations of the same entity increase , the problem becomes more difficult and the system can begin to play the role of the long-term object-based memory ( OBM ) for an autonomous agent . The engineering approach to constructing such an OBM requires many design choices , including the specification of a latent state-space for object hypotheses , a model relating observations to object states , another model specifying the evolution of object states over time , and thresholds or other decision rules for choosing , for a new observation , whether to associate it with an existing hypothesis , use it to start a new hypothesis , or discard it . In any particular application , the engineer must tune all of these models and parameters to build an OBM that performs well . This is a time-consuming process that must be repeated for each new application . In this paper , we develop a method for training neural networks to perform as OBMs for dynamic entity monitoring . In particular , we train a system to construct a memory of the objects in the environment , without explicit models of the robot ’ s sensors , the types of objects to be encountered , or the patterns in which they might move in the environment . Although it is possible to train an unstructured recurrent neural network ( RNN ) to solve this problem , we find that building in some aspects of the structure of the OBM allows faster learning with less data and enables the system to address problems with a longer horizon . We describe a neural-network architecture that uses self-attention as a mechanism for data association , and demonstrate its effectiveness in several illustrative problems . We first validate that OBM can estimate object states when observations are drawn online from a set of cluster centers . Next , we validate that OBM can estimate object states when observations correspond to high-dimensional images . Finally , we illustrate its application on a realistic simulated robotic domain . 2 RELATED WORK . Online clustering methods In the simple setting , where object state does not change over time , the entity monitoring problem can be seen as a form of online clustering , where the assignment of data points to clusters is done online , with observations arriving sequentially and a cumulative set of hypotheses output after each observation . One of the most fundamental online clustering methods is vector quantization , articulated originally by Gray ( 1984 ) and understood as a stochastic gradient method by Kohonen ( 1995 ) . It initializes cluster centers at random and assigns each new observation to the closest cluster center , and updates that center to be closer to the observation . We show that our approach can learn to outperform this online clustering method . More recent work has explored theoretical aspects of online clustering with guarantees ( Liberty et al. , 2016 ; Bhaskara and Rwanpathirana , 2020 ; Cohen-Addad et al. , 2021 ) . Data-Association Filters The most classic filter , for the case of a single entity , is the Kalman filter ( Welch and Bishop , 2006 ) . In the presence of data-association uncertainty the Kalman filter can be extended by considering assignments of observations to multiple existing hypotheses in a DAF or ML-DAF . These approaches , all of which require hand-tuned transition and observation models , are described by ( Bar-Shalom et al. , 2009 ) . We show that our approach can learn the underlying transition and observation models and performs comparably to ML-DAF with ground truth system dynamic and observation models . Visual data-association methods A special case of the entity monitoring problem where observations are closely spaced in time has been extensively explored in the visual object tracking setting ( Luo et al. , 2014 ; Xiang et al. , 2015 ; Bewley et al. , 2016 ; Frossard and Urtasun , 2018 ; Brasó and Leal-Taixé , 2020 ) . In these problems , there is typically a fixed visual field populated with many smoothly moving objects . This enables some specialized techniques that take advantage of the fact that the observations of each object are typically smoothly varying in space-time , and incorporate additional visual appearance cues . In contrast , in our setting , there is no fixed spatial field for observations and they may be temporally widely spaced , as would be the case when a robot moves through the rooms of a house , encountering and re-encountering different objects as it does so . While work has studied the detection of repeated objects with similar appearance ( Girdhar and Ramanan , 2019 ; Bai et al. , 2019 ; Bansal et al. , 2021 ; He et al. , 2021 ; Zhang et al. , 2021b ; a ; Huang et al. , 2019 ) , our focus is on aggregating and estimating the individual states of objects based on substantially different observations in a different space , and our methods are not competitive with specialized techniques on the much more specialized problems of fixed-visual-field tracking or object re-identication . Learning for data association There is relatively little work in the area of learning for generalized data association , but Liu et al . ( 2019 ) provide a recent application of LSTMs ( Hochreiter and Schmidhuber , 1997 ) to a rich version of the data association problem , in which batches of observations arrive simultaneously , with a constraint that each observation can be assigned to at most one object hypothesis . The sequential structure of the LSTM is used here not for recursive filtering , but to handle the variable numbers of observations and hypotheses . It is assumed that Euclidean distance is an appropriate metric and that the observation and state spaces are the same . Milan et al . ( 2017 ) combine a similar use of LSTM for data association with a recurrent network that learns to track multiple targets . It learns a dynamics model for the targets , including birth and death processes , but operates in simple state and observation spaces . Slot Based and Object Centric Learning Our approach to the dynamic entity monitoring task relies on the use of attention over a set of object hypothesis slots . Generic architectures for processing such slots can be found in ( Shi et al. , 2015 ; Vinyals et al. , 2015 ; Lee et al. , 2018 ) , where we use ( Lee et al. , 2018 ) as a point of comparison for OBM . We note that these architectures provide generic mechanisms to process sets of inputs , and lack the explicit structure from DAF we build into our model . Our individual hypothesis slots correspond to beliefs over object hypotheses , and thus relates to existing work in object-centric scene learning . Such work has explored the discovery of factorized objects from both static scenes ( Burgess et al. , 2019 ; Greff et al. , 2019 ; Locatello et al. , 2020 ) . Developed concurrently and most similar to our work , ( Locatello et al. , 2020 ) also utilizes slots as a means of representing a factorized object decomposition of static images . In contrast to ( Locatello et al. , 2020 ) , our work focuses on the use of a set of slots to represent the evolution of uncertain object hypotheses over time , and incorporates attention and inductive biases from DAF to selectively update beliefs across time to obtain object hypotheses as well as their associated confidences . Algorithmic priors for neural networks One final comparison is to other methods that integrate algorithmic structure with end-to-end neural network training . This approach has been applied to sequential decision making by Tamar et al . ( 2016 ) , particle filters by Jonschkowski et al . ( 2018 ) , and Kalman filters by Krishnan et al . ( 2015 ) , as well as to a complex multi-module robot control system by Karkus et al . ( 2019 ) . The results generally are much more robust than completely hand-built models and much more sample-efficient than completely unstructured deep-learning . We view our work as an instance of this general approach . 3 PROBLEM FORMULATION . We formalize the process of learning an object-based memory system ( OBM ) . Formally , when the OBM is executed online , it receives a stream of input observations z1 , . . . zT where zt ∈ Rdz , and after each input zt , it will output two vectors representing a set of predicted properties of hypothesized objects yt = [ ytk ] k∈ ( 1 .. K ) and an associated confidence score for each hypothesis , ct = [ ctk ] k∈ ( 1 .. K ) , where ytk ∈ Rdy , ctk ∈ ( 0 , 1 ) . To ensure that confidences are bounded , we constrain ∑ k ctk = 1 . We limit the maximum number of hypothesis “ slots ” in advance to K. Dependent on the application , the z and y values may be in the same space with the same representation , but this is not necessary . We have training data representing N different entity-monitoring problem instances , D = { ( z ( i ) t , m ( i ) t ) t∈ ( 1 .. Li ) } i∈ ( 1 .. N ) , where each training example is an input/output sequence of length Li , each element of which consists of a pair of input z and m = { mj } j∈ ( 1 .. J ( i ) t ) , which is a set of nominal object hypotheses representing the true current state of objects that have actually been observed so far in the sequence . It will always be true that m ( i ) t ⊆ m ( i ) t+1 and J ( i ) t ≤ K because the set of objects seen so far is cumulative . Our objective is to train a recurrent computational model to perform as an OBM effectively in problems that are drawn from the same distribution over latent domains as those in the training set . To do so , we formulate a model ( described in section 4 ) with parameters θ , which transduces the input sequence z1 , . . . , zL into an output sequence ( y1 , c1 ) , . . . , ( yL , cL ) , and train it to minimize the following loss function : L ( θ ; D ) = N∑ i=1 Li∑ t=1 Lobj ( y ( i ) t , m ( i ) t ) + Lslot ( y ( i ) t , c ( i ) t , m ( i ) t ) + Lsparse ( c ( i ) t ) . ( 1 ) The Lobj term is a chamfer loss ( Barrow et al. , 1977 ) , which looks for the predicted yk that is closest to each actual mj and sums their distances , making sure the model has found a good , high-confidence representation for each true object , with 1 : Lobj ( y , c , m ) = ∑ j min k 1 ck + ‖yk −mj‖ . The Lslot term is similar , but makes sure that each object the model has found is a true object , where we multiply by ck to not penalize for predicted objects in which we have low confidence : Lslot ( y , c , m ) = ∑ k min j ck‖yk −mj‖ . Finally , the sparsity loss discourages the model from using multiple outputs to represent the same true object , by encouraging sparsity in object hypothesis confidences ( derivation in Section D ) : Lsparse ( c ) = − log‖c‖ .

Summary: The paper defines an entity-monitoring problem where the goal is to identify the distinct objects see in an episode where the agent/model moves through the scene/observes partial state. The paper proposes the OBM-Net model architecture to address this problem and identify all the distinct objects observed over each episode. The key idea of OBM-Net to have fixed set of slots which allow tracking multiple hypothesis over time and use an attention mechanism to update the slots with observations over time.