Year: 2021

Abstract:
We propose a framework to continuously learn object-centric representations for visual learning and understanding. Existing object-centric representations either rely on supervisions that individualize objects in the scene, or perform unsupervised disentanglement that can hardly deal with complex scenes in the real world. To mitigate the annotation burden and relax the constraints on the statistical complexity of the data, our method leverages interactions to effectively sample diverse variations of an object and the corresponding training signals while learning the object-centric representations. Throughout learning, objects are streamed one by one in random order with unknown identities, and are associated with latent codes that can synthesize discriminative weights for each object through a convolutional hypernetwork. Moreover, re-identification of learned objects and forgetting prevention are employed to make the learning process efficient and robust. We perform an extensive study of the key features of the proposed framework and analyze the characteristics of the learned representations. Furthermore, we demonstrate the capability of the proposed framework in learning representations that can improve label efficiency in downstream tasks. Our code and trained models will be made publicly available.

heading: 1 INTRODUCTION
text: What are human infants and toddlers learning while they are manipulating a discovered object? And, how do such continual interaction and learning experiences, i.e., objects are discovered and learned one by one, help develop the capability to understand the scenes that consist of individual objects? Inspired by these questions, we aim for training frameworks that enable an autonomous agent to continuously learn object-centric representations through self-supervised discovery and manipulation of objects, so that the agent can later use the learned representations for visual scene understanding.
A majority of object-centric representation learning methods focus on encoding images or video clips into disentangled latent codes, each of which explains an entity in the scene, and together they should reconstruct the input. However, without explicit supervision and more sophisticated inductive biases beyond parsimony, the disentanglement usually has difficulties aligning with objects, especially for complex scenes. We leverage the fact that an autonomous agent can actively explore the scene, and propose that the data collected by manipulating a discovered object can serve as an important source for building inductive biases for object-level disentanglement.
In our proposed framework, whenever an object is discovered by the agent, a dataset containing images and instance masks of this object can easily be sampled via interaction compared to annotating all the objects. Theoretically speaking, any function of the images induced by the discovered object could be a representation of the object. For example, let ϕ be an encoder implemented by a neural network, and let x be the image of an object, we can say that ϕ(x) is a representation of the object. Similarly, the encoder itself can also be a representation of this object since ϕ = argminϕ L(ϕ, x), i.e., ϕ is the output of an optimization procedure that takes the object’s images as input.
We employ network weights as the object-centric representations. Specifically, the proposed method learns an object-centric representation from the data collected by manipulating a single object, through learning a latent code that can be translated into a neural network. The neural network is produced by a discriminative weight generation hypernetwork and is able to distinguish the represented object from anything else. In order to learn representations for objects that stream in one
by one, the proposed framework is augmented with an object re-identification procedure to avoid learning seen objects. Moreover, we hypothesize that object representations are embedded in a low-dimensional manifold, so the proposed framework first checks whether a new object can be represented by learned objects; if not, the new object will be learned as a base object serving the purpose of representing future objects, thus the name object pursuit. Furthermore, the proposed framework deals with the catastrophic forgetting of learned object representations by enforcing the hypernetwork to maintain the mapping between the learned representations and their corresponding network weights.
In summary, our work makes the following contributions: 1) we propose a novel framework named object pursuit that can continuously learn object-centric representations using training data collected from interactions with individual objects, 2) we perform an extensive study to understand the pursuit dynamics and characterize its typical behaviors regarding the key design features, and 3) we analyze the learned object space, in terms of its succinctness and effectiveness in representing objects, and empirically demonstrate its potential for label efficient visual learning.

heading: 2 RELATED WORK
text: Object-centric representation learning falls in the field of disentangled representation learning (Higgins et al., 2016; Kim & Mnih, 2018; Press et al., 2019; Chen et al., 2018b; Karras et al., 2019; Li et al., 2020; Locatello et al., 2020a; Zhou et al., 2021). However, object-centric representations require that the disentangled latents correspond to objects in the scene. For example, (Eslami et al., 2016; Kosiorek et al., 2018) model image formation as a structured generative process so that each component may represent an object in the generated image. One can also apply inverse graphics (Yao et al., 2018; Wu et al., 2017) or spatial mixture models (Greff et al., 2017; 2019; Engelcke et al., 2020b) to decompose images into interpretable latents. Monet (Burgess et al., 2019) jointly predicts segmentation and representation with a recurrent variational auto-encoder. Capsule autoencoders (Kosiorek et al., 2019) are proposed to decompose images into parts and poses that can be arranged into objects. To deal with complex images or scenes, (Yang et al., 2020; Bear et al., 2020) employ motion to encourage deomposition into objects. Besides motion, (Klindt et al., 2021) shows that the transition statistics can be informative about objects in natural videos. Similarly, (Kabra et al., 2021) infers object latents and frame latents from videos. Slot-attention (Locatello et al., 2020b; Jiang et al., 2020) employs the attention mechanism that aggregates features with similar appearance, while Giraffe (Niemeyer & Geiger, 2021) factorizes the scene using neural feature fields. Even though better performance is achieved with more sophisticated network designs, scenes with complex geometry and appearance still lag. As shown in (Engelcke et al., 2020a), the reconstruction bottleneck has critical effects on the disentanglement quality. Instead of relying on reconstruction as a learning signal, our work calls for interactions that stimulate and collect training data from complex environments.
Rehearsal-based continual learning. In general, continual learning methods can be divided into three streams: rehearsal-based, regularization-based, and expansion-based. The rehearsal-based method manages buffers to replay past samples, in order to prevent from forgetting knowledge of the preceding tasks. The regularization-based methods learn to regularize the changes in parameters of the models. The expansion-based methods aim to expand model architectures in a dynamic manner. Among these three types, rehearsal-based methods are widely-used due to their simplicity and effectiveness (Lüders et al., 2016; Kemker & Kanan, 2017; Rebuffi et al., 2017; Cha et al., 2021; von Oswald et al., 2019; Riemer et al., 2018; Lopez-Paz & Ranzato, 2017; Buzzega et al., 2020; Aljundi et al., 2019; Chaudhry et al., 2020; Parisi et al., 2018; Lopez-Paz & Ranzato, 2017). Samples from previous tasks can either be the data or corresponding network activations on the data. For example, (Shin et al., 2017) proposes a dual-model architecture where training data from learned tasks can be sampled from a generative model and (Draelos et al., 2017; Kamra et al., 2017) propose sampling in the output space of an encoder for training tasks relying on an auto-encoder architecture. ICaRL Rebuffi et al. (2017) allows adding new classes progressively based on the training samples with a small number of classes, while (Pellegrini et al., 2020; Li & Hoiem, 2017) store activations volumes at some intermediate layer to alleviate the computation and storage requirement. Co2L (Cha et al., 2021) proposes continual learning within the contrastive representation learning framework, and (Balaji et al., 2020) studies continual learning in large scale where tasks in the input sequence are not limited to classification. Similar to the forgetting prevention component in our framework, von
Oswald et al. (2019) applies a task-conditioned hypernetwork to rehearse the task-specific weight realizations. Please refer to (Parisi et al., 2019; Delange et al., 2021) for a more comprehensive review on this subject.
Hypernetwork. The goal of hypernetworks is to generate the weights of a target network, which is responsible for the main task (Ha et al., 2016; Krueger et al., 2017; Chung et al., 2016; Bertinetto et al., 2016; Lorraine & Duvenaud, 2018; Sitzmann et al., 2020; Nirkin et al., 2021). For example, (Krueger et al., 2017) proposes Bayesian hypernetworks to learn the variational inference in neural networks and (Bertinetto et al., 2016) proposes to learn the network parameters in one shot. HyperSeg (Nirkin et al., 2021) presents real-time semantic segmentation by employing a U-Net within a U-Net architecture, and (Finn et al., 2019) applies hypernetwork to adapt to new tasks for continual lifelong learning. Moreover, (Tay et al., 2020) proposes a new transformer architecture that leverages task-conditioned hypernetworks for controlling its feed-forward layers, whereas (Ma et al., 2021) proposes hyper-convolution, which implicitly represents the convolution kernel as a function of kernel coordinates. Hypernetworks have shown great potential in different meta-learning settings (Rusu et al., 2018; Munkhdalai & Yu, 2017; Wang et al., 2019), mainly due to that hypernetworks are effective in compressing the primary networks’ weights as proved in (Galanti & Wolf, 2020).

heading: 3 METHOD
text: We consider an agent that can explore the environment and manipulate objects which are discovered in an unknown order. Suppose there are N objects in the scene, each of which randomly appears in an image x ∈ RH×W×3, whose ground-truth instance segmentation mask is y ∈ RH×W×N . One can train a deep neural network that maps an image x to its mask y with a dataset D = {(xi, yi)} that consists of such paired training samples. However, sampling from the joint distribution p(x, y) can be extremely time-consuming, e.g., someone may have to manually draw the instance masks for every object in an image.
On the other hand, sampling from the marginals can be much more accessible through interactions. Let Dk be the dataset collected by observing an image xi and the corresponding binary mask of the k-th object yki ∈ RH×W , i.e., Dk = {(xi, yki )} ∼ p(x, yk), which is the marginal distribution obtained by integrating out other objects’ masks in y. The goal of the proposed object pursuit framework is to learn object-centric representations from the data collected by continuously sampling the marginals. Next, we detail the representations used for objects (as illustrated in Fig. 1), and how we can learn them without catastrophic forgetting.

heading: 3.1 REPRESENTING OBJECTS VIA DISCRIMINATIVE WEIGHT GENERATION
text: In order to represent an object, one can compute any functions of the data produced with this object. For example, the encoding of an image containing a specific object that can be used to reconstruct the input image. Here we take a conjugate perspective instead of asking the representation to store information of an object that is good for reconstruction. We propose that the object-centric representation of an object shall generate the mechanisms for performing certain downstream tasks on this object, e.g., distinguishing this object from the others.
Let ϕ be a segmentation network with learnable weights θ that maps an image to a binary mask, i.e., ϕ : Θ × RH×W×3 → RH×W . Moreover, let ψ : ζ → Θ be the mapping from the latent space ζ to the weights of the segmentation backbone ϕ. We define the object-centric representation of an object o as a latent zo ∈ ζ, such that:
E(xi,yoi )∼p(x,yo)∆(ϕ(ψ(zo), xi), y o i ) ≥ τ, (1)
where the expectation is computed according to p(x, yo), i.e., the marginal distribution of object o, and ∆ is a similarity measure between the prediction from ϕ and the sampled mask yo. In other words, zo is a representation of object o, if the network weights generated from zo are capable of predicting high-quality instance masks regarding the object under the corresponding marginal distribution. The threshold τ is a scalar parameter that will be studied in the experiments. Now we detail the proposed object pursuit framework, which unifies object re-identification, succinctness of the representation space, and forgetting prevention, for continuously learning object representations.

heading: 3.2 OBJECT PURSUIT
text: Given the definition of object-centric representations in Eq. 1, our goal is to construct a lowdimensional manifold to embed objects in the input space ζ of the weight generation hypernetwork ψ. We conjecture that the low-dimensional manifold can be spanned by a set of base object representations. More explicitly, we instantiate two lists z and µ, which store the representations of the base objects and the embeddings of the learned objects, respectively. We denote zt−1 = {zi}mi=1 and µt−1 = {µi}ni=1 (n ≥ m, with n the number of learned objects and m the number of base objects, up to time t− 1) as the constructed lists after encountering a (t− 1)-th object. Note that µi has the same dimension as the number of base object representations. Similarly, we denote ψt−1 as the corresponding hypernetwork parameters.
As discussed, when the t-th object ot is discovered, a dataset Dt = {(xj , ytj)} can be easily sampled from the marginal distribution p(x, yt) through interactions. However, such object might already be seen previously. Thus, it is necessary to apply re-identification to avoid repetitively learning the same object. According to the definition in Eq. 1, object ot will be claimed as a seen or learned object if the following condition is true (| · | is the cardinality of a set):
max i≤|µt−1|
E(xj ,ytj)∈Dt∆(ϕ(ψ t−1(zi), xj), y t j) ≥ τ. (2)
with zi = µi ·zt−1. In this case, object ot will be assigned the identity i∗ that achieves the maximum value. Otherwise, if Eq. 2 is not valid, ot is considered as an object that has not been learned.
Learning base object representations. An object ot that can not be identified with the list of learned objects µt−1 can potentially serve as a base object whose representation should be added to the list of base representations z. To ensure that object ot qualifies as a base object, we propose the following test which checks whether ot can be embedded in the current manifold spanned by zt−1:
µ∗ = argmax µ∈R|zt−1| E(xj ,ytj)∈Dt∆(ϕ(ψ t−1(µT zt−1), xj), y t j) + α∥µ∥1, (3)
where µ∗ is the optimal embedding for object ot regarding zt−1 under the ℓ1 regularizer to encourage sparsity. If the first term of Eq. 3 passes the threshold τ with the representation µ∗T zt−1, then we consider ot as an object that should not be added to the list of bases since it can already be represented by the existing base objects.
Next, if ot does not fall on the manifold spanned by zt−1, a joint learning of the representation of ot and the hypernetwork ψ shall be performed so that a new base object representation can be added
to the list. However, since updating the hypernetwork could result in catastrophic forgetting of the previously learned object representations, it is also necessary to constrain the learning process, and the training loss is:
z∗, ψ∗ = argmax z,ψ E(xj ,ytj)∈Dt∆(ϕ(ψ(z), xj), y t j) + α∥z∥1
+ β ∑
i≤|µt−1|
∥ψ(µTi zt−1)− ψt−1(µTi zt−1)∥1, (4)
where the first two terms help to find a good representation for object ot under the sparsity constraint, and the third term enforces that the updated weight generation hypernetwork maintains the previously learned object representations. The value of the negative scalar coefficients α, β will be detailed in the experiments.
Backward redundancy removal. The last but not the least component of the proposed object pursuit framework is to have a backward redundancy check. Since the weight generation hypernetwork is updated to ψt = ψ∗ with Eq. 4, there may now exist an embedding µ∗ (computed using Eq. 3) that re-certifies object ot as an object falls on the manifold spanned by zt−1 under ψt. If this is true, we set zt = zt−1, otherwise, z∗ is added to the list of base object representations since object ot is now confirmed as a base object. In some rare cases, object ot might be hard to learn, e.g., z∗ may not satisfy the criterion described in Eq. 1 under the current hypernetwork ψt. In this case, we simply toss away this object so that it can be better learned in the future as the pursuit process evolves. The proposed object pursuit framework is also summarized in Algorithm. 1.

heading: 4 EXPERIMENTS
text: We target the learning scenario where a scene consists of multiple objects, each of them can be discovered and manipulated through interactions. The objects are learned one by one in a continuous manner but with unknown orders. There are two main aspects of the whole pipeline, i.e., data collection by sampling the marginals of individual objects and construction of the object-centric representations with Object Pursuit. We focus on continuous object-centric representation learning, and thus orient our study on the behavior and characteristics of the proposed object pursuit algorithm. We also perform experiments on one-shot and few-shot learning, and show the potential of the learned object-centric representations in effectively reducing supervisions for object detection. Next, we brief our data collection process.

heading: 4.1 SETUP
text: Data collection. To learn diverse objects from variant positions and viewing angles, we collect synthetic data within the iThor environment ((Kolve et al., 2017)), which provides a set of interactive objects and scenes, as well as accurate modeling of the physics. We collect data of 138 different objects to generate their images and masks. The 138 objects are divided into 52 pretraining objects, 60 train objects for the pursuit process, and 25 test unseen objects. To focus on the representation learning part, we abstract the interaction policy, and the data collection procedure of a single object can be summarized as follows: 1) Randomly set the positions of all the objects in the scene. 2) Calculate all available camera positions and viewing angles from which the target object (to be learned) is visible so that the sampling is effective. The camera position, yaw angle, and pitch angle change within the range of 0.4 (grid size), 4◦ and 30◦ respectively. 3) For each camera position and viewing angle, we collect a 572× 572 RGB image and a binary mask of the target object. 4) Repeat (1-3) for all objects in the stream. Please check Fig. 2 for the sampled data.
Table 1: Re-identification: recall and precision on seen objects.
τ 0.5 0.6 0.7 0.8
recall 1.0 1.0 1.0 1.0 precision 1.0 1.0 1.0 1.0
Table 2: Re-identification: rate of unseen objects been identified along the course of the pursuit process.
τ No. of trained objects
8 16 24 32 40 48 56
0.5 0.40 0.52 0.56 0.60 0.64 0.64 0.72 0.6 0.08 0.20 0.28 0.44 0.56 0.60 0.60 0.7 0.16 0.28 0.32 0.40 0.40 0.48 0.44 0.8 0.00 0.08 0.16 0.24 0.28 0.28 0.28
Network implementation In our experiment, we use Deeplab v3+ (Chen et al., 2018a) as the segmentation network ϕ, which consists of 3 parts: a backbone to encode features at different levels, an aspp module, and a decoder to predict the segmentation probability per pixel. We use resnet18 as the backbone (encoder), whose weights are fixed both in the pretraining and the pursuit process. The weights of the aspp module and the decoder are generated by the convolutional hypernetwork ψ. For each convolution layer in the aspp module and the decoder, ψ takes object representation z as input, and predict weights of the convolution kernel using an upsampling convolution block. The input representation z first expanded to a 1024-dim vector by a linear mapping and resized to a 1 × 1 × 32 × 32 tensor. After going through several upsampling blocks, each of which consists of an upsampling followed by a convolution and a leaky Relu, the 1 × 1 × 32 × 32 tensor turns into the output kernel weight. For other network weights like ’running mean’ and ’running var’ in batch normalization, the hypernetwork linearly maps representation z to generate them.
Training details. For the similarity measure ∆, we use the dice score proposed in (Milletari et al.). In addition to ∆, we find that it will be beneficial to add an extra binary cross-entropy term when learning base object representations using Eq. 4. Note, to deal with imbalanced foreground and background sizes, we also put a weighting on the entropy terms that correspond to the object so the learning can be more efficient. The sparsity constraint α is set to −0.2,−0.1 for Eq. 3 and Eq. 4 respectively, and β = −0.04 for all our experiments. To improve the convergence, we also warm up the hypernetwork using the pretraining objects. During pretraining, each mini-batch contains training data from one object, and we randomly choose which object to use in the next batch. In backpropagation, we update the hypernetwork ψ and representation z for each object. When the pretraining is done, we perform a redundancy check to get rid of the objects that can be represented by others. For simplicity, this check is performed in sequential order, and we are left with a set of base object representations to carry out the following studies.

heading: 4.2 ON THE REPRESENTATION QUALITY MEASURE
text: The learning dynamics and the output of Algorithm. 1, i.e., the lists of base object representations z and the learned objects µ, together with the weight generation hypernetwork ψ, are primarily affected by the representation quality measure τ introduced in Eq. 1. For example, τ controls whether an object will be claimed as seen, and it also determines whether or not an object falls on the manifold spanned by the current base object representations. We study each of them in the following.

heading: 4.2.1 RE-IDENTIFICATION
text: As described in Eq. 2, when an object is discovered, it will be first checked against the learned objects and re-identified if the maximum expected similarity passes τ . To examine how the quality measure τ influences the re-identification process, we run multiple object pursuit processes with different τ ’s. All runs are performed with the same training object order so that the only variant is the value of τ . For evaluation, we preserve a separate set of 25 objects (unseen test objects) that never appear during training. Note, among these unseen test objects, there are also objects that are similar to the training ones. And we use 27 objects (seen test objects) from the warp-up joint training described above to check the re-identification accuracy.
First, we check how τ affects the re-identification for seen objects. As reported in Tab. 1, if an object is learned and added to the object list µ, it will be claimed as seen by Eq. 2, and the re-identification accuracy is always one. This is true for τ varying between 0.5 and 0.8, which demonstrates the robustness of the re-identification process against τ for objects learned.
Second, we check the behavior of the re-identification component for unseen objects under different τ ’s along the pursuit process. In Tab. 2, we can observe that as more and more objects are learned during the pursuit, the unseen objects that are claimed as seen from the re-identification process also increase. This observation is consistent across different τ ’s. Furthermore, the rate of unseen objects identified as seen converges at the end of the pursuit process, but at different levels for different τ ’s, i.e., the converged rate is lower for larger τ . It may seem incorrect if an unseen object is claimed as seen by the reidentification component. However, if we examine the unseen objects (see Fig. 3), we can see that it is quite natural for these unseen objects to be labeled as seen, because they are similar to one or multiple objects in the object list µ. This is indeed a desired characteristic since representing
or learning an object that is similar to existing ones may not be informative. Moreover, one can adjust τ to tune the similarity level. For example, if one insists on learning an object similar to previously seen objects, increasing the value of τ should work as evidenced by the converged rates for τ ’s in Tab. 2.
In a nutshell, the representation quality measure τ has little effect on the re-identification recall and accuracy for learned objects. Yet, it controls the granularity of the learned representations by modulating the rate of unseen objects that would be identified as learned ones.
4.2.2 SUCCINCTNESS AND EXPRESSIVENESS
We want to study how the representation quality measure τ affects the overall learning dynamics in terms of the succinctness and expressiveness of the learned base object representations. By checking Eq. 2, Eq 3 and also the previous experiment, we conjecture that if τ is small, objects similar to the learned ones will be more easily identified as seen and certified as on the manifold. If so, the number of objects that will be used for learning the base representations may also be small, thus increasing the succinctness of the final representations. Conversely, when τ increases, we would expect that more objects will contribute to the base representations, thus increasing the expressiveness. We like to check if the observations align with our conjecture and how such behav-
ior affects the quality of the learned bases. To facilitate the analysis, we propose to check the following quantities: 1) |z|/N , which is the portion of objects that contribute to base representations; 2) |µ|/N , which is the portion of learnable objects that are added to the object list µ; 3) Re, rate of objects that are confirmed unseen but can be expressed by the base object representations; 4) Rf , rate of objects to be learned as base representations, which are later considered as redundant or unqualified; 5) Aµ, segmentation accuracy on learned objects. We report the above metrics across different τ ’s in Tab. 3. As expected, a larger τ generally encourages more objects to be learned as base representations. For example, the number of base objects learned is much larger when τ increases from 0.5 to 0.6 (first row). This is also evidenced by the third row of Tab. 3, which shows that the probability of an unseen object to be expressed by base rep-
resentations will decrease as τ increases, creating more attempts to learn objects as bases. However, the number of learnable objects, i.e., a base object or an object that falls on the manifold spanned by the bases, attains the maximum at a medium value τ = 0.6 (second row). The underlying reason is two-fold: First, a very small τ means that many objects will be identified as seen and thus discarded to save computation; Second, a very large τ can make the qualification of an object representation extremely difficult such that it will be put aside for future learning. The latter is also supported by the metric Rf shown in the fourth row, i.e., the probability that an object will be considered redundant or unqualified after learning as a base object will increase as τ becomes large. Lastly, when checking the quality of the base representations in expressing a common set of learned objects, we can see that the segmentation accuracy correlates with τ in a positive manner (fifth row).
In general, τ directly impacts the quality of the base representations for learned objects, but its effect on the number of base representations produced by the pursuit procedure is not monotone. Within a moderate range, we can increase τ to encourage learning more base representations, however, we may not want τ to be too large that only a few objects are qualified as base representations.

heading: 4.2.3 LABEL EFFICIENCY
text: Besides the training dynamics, we evaluate the usefulness of the learned object base representations in terms of how it facilitates learning the representation of a new object with only a few annotations. For comparison, we also perform learning of the object representations over the entire representation space. Training is similar to Eq. 3. The quality of the learned object representations is measured by their segmentation accuracy on test data.
As reported in Tab. 4, the quality of the few-shot learned representations increases as τ gets large, which aligns with our observation in the previous section that the expressiveness of the learned object base representations highly correlates with τ . However, note that there is a slight drop in performance when τ increases from 0.7 to 0.8 (fourth and fifth column). The reason is that as τ gets really large, it also becomes much easier to omit objects that can not pass the quality test. As a result, the hypernetwork, which translates the representation to network weights, also gets less trained. Thus, when tested on new objects, the performance may not match that of the trained objects for the same set of base representations, suggesting again that a moderate τ is needed to balance between the succinctness and generalization of the learned base representations.
The above observation does not hold for the representations learned over the full space. Moreover, when comparing the performance within the low data regime, we can see that those object representations found on the manifold outperform those found in the entire space by a large margin. For example, the new object representations found with the learned bases under τ = 0.7 outperform their counterparts by 57.6%, 35.5%, 29.2% for the 1-shot, 5-shot, and 10-shot settings, respectively. This demonstrates the potential of using the learned base representations to help reduce the supervision needed to learn a new object. Also, it confirms that the learned base representations are meaningful since the manifold spanned by them provides a good regularity for learning unseen objects.
4.3 ORDER OF TRAINING OBJECTS
The proposed object pursuit algorithm learns object representations in a stream, so we also check how the learning dynamics vary when the order of training objects changes. We fix τ to 0.6 and run ten pursuit processes with random training object order. We reported the mean and standard deviation of the metrics proposed in Tab. 3. As observed in Tab. 5, the pursuit process is robust to the training object order.
4.4 FORGETTING PREVENTION
In this section, we want to check if the forgetting prevention term in Eq. 4 is effective and how it affects the pursuit dynamics. We run pursuit processes with different values of the coefficient β, where the quality measure τ and training object order is fixed. Segmentation accuracy Aµ and forgetting rate γf (i.e., how many objects falls under the quality measure after the process is finished) in Tab. 6 demonstrate the effectiveness of the forgetting prevention term: when |β| decreases, the segmentation accuracy drops, and the forgetting rate increases; when |β| vanishes, the forgetting rate reaches 97%, which means that the hypernetwork almost forgets all the object representations it previously learned. We can also observe that both |z|/N and |µ|/N increase
when |β| decreases. This is due to the fact that when the hypernetwork forgets what are learned, any incoming object will be unlikely to be considered as seen, nor to be expressed by current bases. So the hypernetwork tends to learn them as new base objects, which causes |z|/N to increase. This is also evidenced by the drop in Re, which is rate of new objects that are certified as on the object manifold. Furthermore, without the constraint of the forgetting prevention term, it is more likely to get higher accuracy in learning a new object, which decreases the number of unqualified objects. Since the number of redundant objects and unqualified objects both drop when |β| decreases, |µ|/N increases. Thus, in order to reduce computational cost and enforce learning meaningful representations, one would like to apply a relatively large |β| during the pursuit process. We can also observe that as |β| changes from 0.02 to 0.1, Rf increases monotonically, this is because the forgetting prevention constraint affects the quality of the learned representations, since less freedom is available when |β| is extremely large. Consequently, fewer objects will be qualified with a good representation measured by τ . On the other hand, Rf is also high when beta is set to 0. The reason is that when learning a new object without the constraint of the forgetting prevention term, the hypernetwork tends to overfit, thus making it easier for this new object to be considered as redundant, i.e., it can be expressed by the existing base representations, even though the learned representation will be forgotten by the network right after the current learning episode.

heading: 4.5 MORE RESULTS
text: In the appendix, we also provide ablation studies on how the sparsity constraints in Eq. 3 and Eq. 4 affect the learning dynamics and the quality of the learned representations. By examining the most relevant base objects for a novel object that can be expressed by the base representations (Fig. 4), we can qualitatively see that high-level concepts are learned within the representation space as objects that share similar geometry or appearance will be more correlated than others. For curiosity, we also test the usefulness of the base object representations on real-world video objects. As demonstrated in Fig. 7, the learned base representations can capture well the representations of real-world objects with a single learning example even if they are trained on synthetic data.

heading: 5 CONCLUSION
text: We demonstrate that the proposed object pursuit framework can be used for continuously learning object-centric representations from data collected by manipulating a single object. The key designs, e.g., object re-identification, forgetting prevention, and redundancy check, all contribute to the quality of the learned base object representations. We also show the potential of using the learned objectcentric representations for tasks at a low-annotation regime. Especially, the learned object manifold provides a meaningful and effective prior on objects, which can facilitate downstream tasks that require object-level reasoning. As inspired by an initial attempt on the real-world data (Fig. 7), we would also like to check the proposed object pursuit algorithm in the real world. For example, we can train an autonomous agent to collect data from the natural 3D environment with a more efficient interaction policy, and then test the learned object representations on real-world compositional visual reasoning tasks. These are in our future research agenda.
Ethics Statement. The proposed object pursuit framework aims at learning object representations for object-centric visual reasoning tasks. Currently, the experiments are performed in simulation, which is publicly available and comes with a proper license. However, how to use the learned representations could be an issue, and we will explicitly state the guideline on how to use our code and trained models ethically. In our future research, when data collection in the real world is involved, we will consult the university ethics review committee for advice. However, in the current form, we do not observe any significant concerns.
Reproducibility Statement. Our code, training data, and learned models will be made publicly available after the acceptance of our paper. We will add detailed comments in the code so that the implementation can be easily understood. For a preview of the implementation, please refer to the attached code in the supplementary materials.

heading: A APPENDIX

heading: A.1 ALGORITHM
text: Here is the Object Pursuit algorithm we describe in the method section. In this algorithm, we first check if the object is seen according to Eq. 2. If the object is seen, we directly move on to the next object, otherwise, we start to check if the object can be represented by current bases according to Eq. 3. If it can be expressed, we add this object to the object list µ and move on to the next object, otherwise, we need to train for a new z and update the hypernetwork using loss function Eq. 4. After training, we start a second redundancy check, with the same criterion Eq. 3 in the first check. If the object can be represented by bases, we just add it to the object list µ, otherwise, we consider it a new base and add it to both base list z and object list µ. The algorithm runs in a loop to simulate that objects are continually showing up and learned by our system.
Algorithm 1: Object Pursuit Result: A set of object representations µT = {µi}Ni=1, zT = {zi}Bi=1, and the hypernet ψT , with T ≥ N ≥ B initialization: z0 = µ0 = ∅, ψ0 is randomly initialized; while t ≤ T do
Sample Dt = {(xj , ytj)} ∼ p(x, yt); Check if object ot is in µt−1 (parameter τ ); if YES then
µt = µt−1; zt = zt−1; ψt = ψt−1;
else Check if object ot can be represented using zt−1 (parameter τ ); if YES then
µt = µt−1 ⋃ µot ; zt = zt−1; ψt = ψt−1;
else Training for zot and ψ
′ under the constraint of all objects in µt−1; ψt = ψ′; Check if zot can now be approximated by z
t−1; if YES then
µt = µt−1 ⋃ µot ;
zt = zt−1; else
µt = µt−1 ⋃ [0, 0, ..., 0, 1];
zt = zt−1 ⋃ zot ;
end end
end end

heading: A.2 ABLATION ON THE SPARSITY CONSTRAINTS

heading: 1. L-1 norm coefficient on µ
text: In this ablation study, we aim to understand whether L-1 norm on combination coefficient µ in Eq. 3 affects an object falls on the manifold or not. From Tab. 7 we can see that when L-1 norm coefficient |α| change from 0.0 to 0.2 (absolute value), Re increases, which means more object can be expressed by current bases, causing the number of base objects (|z|/N ) to decrease. It shows that if we properly increase the constraint on µ and promote the sparsity of µ, it may be easier to find a µ that can express the object. However, if |α| exceeds a certain limit, e.g. 0.5, too much constraint added to µ (which makes µ harder to change), finding
a µ to express an object becomes difficult, thus the number of objects that can be expressed (Re) decreases, as shown in Tab. 7.
We can also see that Rf first drops then increases when |α| gets bigger. Our reasoning is that, even though more objects can not be expressed by the bases at the first redundancy check and have to be learned as new representations when |α| increases, the chances that they are unqualified or redundant after training also increase, making the base number continually decreases.
2. L-1 norm coefficient on z
In this ablation study, we focus on how L-1 norm on z affects object pursuit. We change the coefficient α and evaluate the pursuit process. As Tab. 8 shows, when |α| increases, the number of objects that can be expressed by bases (Re) also increases, causing the base number (|z|/N ) to decrease. This is because regularization on z prevents the object representations from getting too far from each other, thus making z distribute more uniformly. Since z is well distributed, it may be easier to find a coefficient µ that can express an object. We also find it in our experiment that when |α| gets bigger, more objects will be considered
as unqualified due to their low training accuracy, especially when |α| = 0.5. It shows that more constraints on z may cause it harder to find a proper z to represent an object during training, thus decrease the accuracy. It also explains why the number of learnable objects (|µ|/N ) decreases when |α| change from 0.1 to 0.5.

heading: A.3 UNDERSTAND THE BASE REPRESENTATIONS
text: Fig. 4 shows some visualization results of unseen objects and the corresponding active base objects. The results are from the pursuit process. When an unseen object is detected, and if it can be expressed by the current bases, we then find the combination coefficient µ through Eq. 3. In Fig. 4, we show three examples that the unseen objects can be expressed by base objects, and the combination coefficient values are shown below the corresponding bases. We show the top 5 bases that have the highest coefficient value among all the bases.
In the first example (the 1st row), the base object ’DishSponge 1’ has the highest coefficient value in expressing a green cup (Cup 3). We conjecture that ’Cup 3’ and ’DishSponge 1’ share a similar color, and green objects are rare in this set of bases, causing the DishSponge’s coefficient to be the highest one. ’Dumbell 1’ may share a similar shape with ’Cup 3’, since they are both thin in the middle and thick in the end. The second example (the 2nd row) shows that if there is a base object (Bowl 7) that is evidently more similar to the target object (Bowl 10) than other bases, its coefficient value may be the highest. Here Bowl 7 and Bowl 10 are similar in both color and shape, but they are different in shape at the bottom of the bowl. In the third example (the 3rd row), the black cup (Cup 2) and the black pot (Pot 1) share the same color, while the black cup (Cup 2) and the glass cup (Cup 1) share a similar shape.
These results show that the object representation space learned through object pursuit characterizes some high-level concepts (e.g. color, shape) that help better understanding the objects. An unseen object tends to be represented by a base object with similar color or shape. Although in some cases, understanding the base representations is not as easy as the examples shown in Fig. 4, this regular pattern is still evident in most cases.

heading: A.4 MORE VISUAL RESULTS ON SYNTHETIC DATA
text: Fig. 5 shows more segmentation results from the experiments. In these examples (the 1st row and the 2nd row), ’bowl’ and ’kettle’ are two unseen objects which are not in the object list µ and the base list z. To represent an unseen object, we fix the hypernetwork and try to find a combination coefficient µ∗ to express the target object with the bases, according to Eq. 3. The unseen object, which resides on the manifold (the object space) with latent code z can be segmented by the segmentation network generated from its representation with the hypernetwork. Note, we test the segmentation network on the validation dataset.
Even if the unseen object is expressed by base objects which may share some similarity with it, the unseen object can still be segmented accurately from various backgrounds in different positions and angles, as Fig. 5 shows. It verifies that the learned representations, together with the hypernetwork, preserve discriminative object-centric knowledge.

heading: A.5 REAL-WORLD DATA COLLECTION AND LEARNING
text: We propose that our framework object pursuit can be used with autonomous agents that explore the environment and interact with objects. Through interactions, an agent could learn object-centric representations, and this is an unsupervised setting since no human annotations are required when the agent interacts with objects in the physical scene.
Currently, we perform the major experiments on synthetic data due to the lack of a robot to perform the data collection in the real world. Moreover, we like to have a proof of concept before carrying out real-world experiments, which could involve an enormous amount of research funds and effort, which is out of the scope of our current work. However, we are confident that transitioning from the synthetic environment to the real world is highly practical as all the technical components that need to be used in the real world are ready.
In order to evaluate how practical it is for a robot to perform the pursuit process in the real world, we need to look into two aspects. First, how possible it is to obtain instance masks of the manipulated object. Second, how efficient the learning can be given the objects that need to be learned. Next, we demonstrate how it is possible to get the instance masks of a real-world object through interaction. And we discuss the second aspect in the following section.

heading: A.5.1 COLLECT DATA IN THE REAL-WORLD VIA INTERACTION
text: In our setting, a robot manipulates only one object at a time and learns its representation. Two cues can be used to obtain the instance mask of the object being manipulated. Robotic arm localization and motion. It is easy to know where the robotic arm is within the view through calibration. Also, motion segmentation is a well-studied topic on real-world objects in the literature (Charig Yang et al., 2021; Yang et al., 2019). A possible pipeline is to first apply motion segmentation on the images, and this should give the masks of both the robotic arm and the object as they are both moving. To remove the portion of the robotic arm, the agent can treat its arm as the first object to learn, so that it knows how to segment its arm in the future. By subtracting the arm, it now has the mask of the manipulated object.
Fig. 6 shows an example of getting instance masks from a video of object manipulation without supervision. In this example, the background in the video is static (but the motion segmentation algorithm we employed can also work when there is ego-motion), and we first segment the cube together with the robot hand using motion segmentation (Charig Yang et al., 2021). Then we remove the robot hand from the segmentation mask, using “densecrf” (Krähenbühl & Koltun, 2011) as post-processing. As mentioned, there are other substitutes to exclude the hands, such as using a specifically trained segmentation model of the robotic arm. The segmentation process can run at 5 frames per second. And the results show the practicality of obtaining instance masks during manipulating objects in the real world.

heading: A.5.2 REAL-WORLD OBJECT PURSUIT
text: To evaluate the efficiency and robustness of learning on real-world images or videos, we conduct experiments on two large-scale real-world datasets, i.e., Youtube VOS (Xu et al., 2018) and CO3D (Reizenstein et al., 2021).
YouTube-VOS We train and evaluate our framework on the Youtube-VOS dataset, which contains 65 categories. Each category may appear in multiple video sequences, and there may be multiple instances of the same category in a video sequence. We traverse all the categories, and for each category, we sample one video sequence from all sequences containing a single instance of this category to serve as the object we pursuit (Note this is to stimulate the interaction that should happen in the real world). The number of frames in a video sequence typically ranges from 20 - 36. For each video sequence, we repeat the frames many times so that the effective dataset size is around 500 samples. For each frame, we apply horizontal flip augmentation and random crop augmentation. Specifically, we crop each frame with the same ratio along the x and y-axis, which is a random number between 0.6 and 1. Finally, we resize the frame to 320x180 (The origin size is 1280x720).
CO3D We also test our framework on CO3D. Since processing the original dataset is too timeconsuming (18,619 objects of 50 MS-COCO categories), we randomly select 285 objects from 8 object classes: apple, banana, backpack, baseball bat, bench, bicycle, book, and bottle as training objects. The data of each object contains a video sequence that shows different views of the object. We randomly crop each frame to a square image, then apply horizontal flip augmentation. After augmentation, the effective dataset size is about 200 samples per object. Each frame is at the resolution 256x256.
Other training details. For both datasets, we initialize the hypernet with the model pretrained on synthetic data, with no initial bases and objects. We set different quality measures τ (0.5, 0.6, 0.7, and 0.8) to evaluate our framework on real-world data. Other settings are the same as the synthetic data experiment.
Pursuit dynamics. Tab. 9 and Tab. 10 show pursuit dynamics under different τ on YouTube-VOS and CO3D datasets, respectively. The real-world data experiments share similar patterns with the synthetic data experiments: when τ gets bigger, Aµ (average segmentation accuracy) and Rf (proportion of unqualified and redundant objects) gets bigger, while Re (proportion of expressed objects) first increases then decreases. A higher τ excludes objects with poor training accuracy, thus improving the overall segmentation performance. On the other hand, a higher τ could make an object easier to be considered as unqualified, which would be skipped and would not be learned by our system. The number of bases and qualified objects both increase with τ , because fewer objects would be claimed as seen when τ increases, increasing the number of learned objects. The similar patterns between real-data and synthetic data show that the conclusions we made from Tab. 3 can generalize to the real-world domain and transfer between different datasets.
Learning efficiency. The running time of our framework depends on the threshold τ . Generally, a smaller τ leads to a shorter running time since objects are easier to be expressed by bases or recognized as seen objects, which will save the time of learning base object representations. In the synthetic data experiments, τ = 0.5 could finish running 67 objects in about 10 hours, while τ = 0.8 needs 1.5 days. In the real-world experiments on youtubeVOS and CO3D datasets, our framework can learn around 80 objects per day under τ = 0.5 and 40 objects per day under τ = 0.8.
Conclusion on real-world practicality. With these experiments, we can conclude that the observations we make in the synthetic environment transfer to real-world data. Moreover, two key factors
that are directly related to how practical it is to run object pursuit in the real world are checked to be positive. First, collecting data with object instance masks through interaction in the real world is practical, as verified by the effectiveness of the proposed pipeline in the previous section. Second, the pursuit framework is robust on the real-world data, and the efficiency is good enough to perform learning in the real world. For example, in a house with hundreds of objects, the training can be finished within three weeks and requires no human supervision, which is far more time-consuming and complicated in practice.

heading: A.6 ADDITIONAL RESULTS ON RE-IDENTIFICATION
text: In section 4.2.1, we demonstrate the impact of the quality measure τ on seen objects and unseen objects separately during re-identification. To further elaborate on the impact of τ , in this section, we report precision and recall on both seen objects and unseen objects (collectively) on the scale of all testing objects. For unseen objects, we define recall as the fraction of correctly identified unseen objects among all the unseen objects, and precision is defined as the fraction of correctly identified unseen objects among all the objects we identify as unseen. Same for seen objects.
Tab. 11 and Tab. 12 report recall and precision on unseen objects and seen objects, taking all testing objects into consideration. We collect the number of objects our model identifies as seen or unseen from the re-identification experiment introduced in Section 4.2.1, then compute recall and precision. From these two tables, the precision of unseen objects and the recall of seen objects are 1.0 under all τ , which shows that a seen object can always be identified correctly. This is crucial to our framework: if a seen object is not identified as seen, it would be problematic since we have to learn that object repeatedly, unlimitedly enlarging the object list µ. On the other hand, the recall of unseen objects and the precision of seen objects are lower than 1.0, showing that some unseen objects could be identified as seen. By visualizing the data (Fig. 3), we find this situation happens only when the unseen object has substantial similarities., e.g., in terms of color or shape, to a seen object. It has no significant impact on the training process, since these “seen but actually unseen” objects contribute very little information to the learned object representations. Therefore, our model learns object representations in a way that prevents learning the same or highly similar objects, improving efficiency.
We also find that the recall of unseen objects and the precision of seen objects get larger when τ gets bigger. It is because a larger τ increases the bar of determining an object as seen, according to Eq. 2. Therefore, a difference would make the model consider two as different objects. As τ gets bigger, the learned representation becomes more discriminative, at the cost of generalization, which is a characteristic that allows us to tune the system depending on the actual need.

heading: A.7 EVALUATION ON BASE AND NON-BASE OBJECTS
text: Tab. 3 shows pursuit dynamics, including segmentation accuracy and re-identification rate with different τ for base and non-base objects. We take all objects in the object list µ into consideration. However, these objects are added to the object list in two different ways: some can be expressed by current bases (non-base), while others are trained and accepted as a new base (base). For the former, we simply find combination coefficients µ without updating the hypernet, optimizing the model in much smaller parameter space than the latter. This may cause different pursuit dynamics in these two situations. This section analyzes the segmentation accuracy and re-identification rate to promote the understanding on this aspect.
As reported in Tab. 13, for both base and non-base objects, Aµ shows the average segmentation accuracy, and Rreid shows the proportion of the objects that are correctly re-identified as seen objects. We report Aµ and Rreid under different τ . For τ = 0.8, we found that all objects that appeared in the pursuit sequence could not be expressed by bases at first and would be learned as a new base, since the threshold τ is too high for an object to be considered as being expressed
by bases. In this case, we could not compute Aµ and Rreid for non-base objects. For other τ , the re-identification rate Rreid stays stably at 1.0, showing that if a base or non-base object has been encountered, it could be re-identified correctly. Tab. 13 shows that Aµ of both base and non-base objects increases when τ gets bigger, which shares a similar pattern with Tab. 3.
We also find that Aµ on non-base objects could be lower than Aµ on base objects. It indicates that updating the hypernet and training as a new base can perform better than simply combining bases due to a larger degree of freedom. This difference can be reduced by increasing the capacity of the hypernetwork. In conclusion, the re-identification performance is stable and accurate on both base and non-base objects, and the segmentation accuracy increases with τ . Furthermore, the segmentation accuracy of base objects is generally higher than non-base objects.

heading: A.8 ONE-SHOT LEARNING ON REAL DATA
text: We perform one-shot learning on the DAVIS 2016 dataset (Perazzi et al., 2016), a video object segmentation dataset in the real scene. Under the one-shot learning scheme, we fix the hypernet and the bases, initialize the combination coefficient µ with only one
training sample (first frame in the sequence). From µ, we can get the representation z for the training object, generate the parameters of a segmentation network using the hypernet, then evaluate the segmentation accuracy. We first conduct object pursuit on DAVIS training set as finetuning, acquiring an updated hypernet and new bases. We then test one-shot learning on the DAVIS evaluation set, which contains 20 video sequences. We use the first frame as initialization and evaluate the rest frames. Finally, we compare our results with a set of video object segmentation works on the DAVIS benchmark, using the Jaccard index (IoU) as the evaluation criterion.
Tab. 14 shows the quantitative evaluation on DAVIS. We report Jaccard Mean (average Jaccard score for all test objects), Jaccard Recall (average Jaccard score for test objects whose score is higher than 50.0), and Jaccard Decay (evaluate the Jaccard score decay by time). We compare our framework with video object segmentation methods that use both appearance and temporal information, such as SEA (Avinash Ra-
makanth & Venkatesh Babu, 2014), HVS (Grundmann et al., 2010) and JMP (Fan et al., 2015). We also compare our work with OSVOS (Caelles et al., 2017), a work of one-shot learning on video without temporal information. To make a fair comparison, we implement OSVOS using the same backbone in our framework, and remove the post-processing. As shown in Tab. 14, our method outperforms SEA, HVS, and JMP on both Jaccard mean and recall. Compared with OSVOS, we have a slightly worse Jaccard mean score and do better on recall score. It shows that our method can reach a better performance on objects that have high segmentation accuracy. Generally speaking, our method is comparable to the state-of-the-art on one-shot object segmentation learning, though there is enough space for us to push the performance.
However, in terms of learning efficiency, our method performs much better, as shown in Tab. 15. Since we only need to optimize the coefficient µ with the size of |z|, the number of optimizable parameters of our method is much smaller than that of OSVOS. So our framework is more efficient in storage if the trained networks need to be transmitted and used elsewhere. Another advantage of our framework is that the one-shot learning is much faster, which shows the potential of using our framework for real-time applications.
Fig. 7 shows some visualization results of one-shot learning on the DAVIS dataset. Although only one sample is provided during one-shot training, our model can predict the masks on subsequent frames. In some sequences, for example, the ’dog’ sequence showed in the 4th row of Fig. 7, the viewing angle and the object’s shape vary significantly between frames, making it challenging to predict subsequent frames only based on the first frame without any object prior. It could be inferred from the results that our model contains useful object-centric priors that help segment objects in subsequent frames.

Review summaries:
This paper presents a new framework to learn object-centric representations. The model is composed of a segmentation network and a hypernetwork. The hypernetwork takes the latent representation of a certain object as input and predicts the weights for the segmentation network. The latent representation and the hypernetwork are jointly optimized to maximize the discrimination power of the segmentation network. The framework also introduces a sparse representation mechanism so that each object can be represented in some base object representations. 

This paper proposed a framework to continuously learn object-centric representations and formulate the problem by projecting the object representation space to the hypernetwork parameters for the segmentation task. The data are sampled from marginals where only one instance mask is collected in each scene. The representation learning incorporates the base representation learning, redundancy removal, and forgetting prevention.  The experiments study the impact of \tau under different metrics.