Original: S TREAMING

Original: I NFERENCE

Original:  FOR I NFINITE N ONS TATIONARY C LUSTERING
Revised: S TREAMING I NFERENCE FOR I NFINITE N ON -S TATIONARY C LUSTERING

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Learning from a continuous stream of non-stationary data in an unsupervised manner is arguably one of the most common and most challenging settings facing intelligent agents.
Revised: Learning from a continuous stream of non-stationary data in an unsupervised manner is arguably one of the most common and most challenging settings facing intelligent agents.

Original: Here, we attack learning under all three conditions (unsupervised, streaming, non-stationary) in the context of clustering, also known as mixture modeling.
Revised: Here, we attack learning under all three conditions (unsupervised, streaming, non-stationary) in the context of clustering, also known as mixture modeling.

Original: We introduce a novel clustering algorithm that endows mixture models with the ability to create new clusters online, as demanded by the data, in a probabilistic and principled manner.
Revised: We introduce a novel clustering algorithm that endows mixture models with the ability to create new clusters online, as demanded by the data, in a probabilistic, time-varying, and principled manner.

Original: To achieve this, we first define a novel stochastic process called the Dynamical Chinese Restaurant Process (Dynamical CRP), which is a non-exchangeable distribution over partitions of a set; then, we show that the Dynamical CRP provides a non-stationary prior over cluster assignments and yields an efficient streaming variational inference algorithm.
Revised: To achieve this, we first define a novel stochastic process called the Dynamical Chinese Restaurant Process (Dynamical CRP), which is a non-exchangeable distribution over partitions of a set; next, we show that the Dynamical CRP provides a non-stationary prior over cluster assignments and yields an efficient streaming variational inference algorithm.

Original: We conclude withpreliminary experiments showing that the Dynamical CRP can be applied on diverse data.
Revised: We conclude with experiments showing that the Dynamical CRP can be applied on diverse synthetic and real data with Gaussian and non-Gaussian likelihoods.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Biological intelligence operates in a radically different data regime than (most) artificial intelligence.
Revised: Biological intelligence operates in a radically different data regime than most artificial intelligence.

Original: Biological intelligence must contend with data that is (i) unsupervised, (ii) streaming, and (iii) nonstationary, either as a consequence of the agent, its environment or both.
Revised: In particular, biological intelligence must contend with data that is (i) unsupervised, (ii) streaming, and (iii) non-stationary, either as a consequence of the agent, its environment, or both.

Original: One goal of open-endedness is to make artificial intelligence significantly more capable in this data regime, and accomplishing that goal requires asking and answering how agents in this data regime ought to approach learning.
Revised: One goal of lifelong learning is to make artificial intelligence significantly more capable in this data regime, and accomplishing that goal requires asking and answering how agents in this data regime ought to approach learning.

Original: In this paper, we consider the specific unsupervised problem of clustering, also known as mixture modeling.
Revised: In this paper, we consider the specific unsupervised problem of clustering, also known as mixture modeling.

Original: Clustering is a ubiquitous and important problem in its own right, with widespread applications, but clustering can also serve as an intermediary sub-goal in service of other goals: for instance, an agent in a partially observable world may wish to cluster sensory observations into world states to then use for planning or reinforcement learning.
Revised: Clustering is a ubiquitous and important problem in its own right, with widespread applications, but clustering can also serve as an intermediary sub-goal in service of other goals: for instance, an agent in a partially observable world may wish to cluster sensory observations into world states to then use for spatial navigation or reinforcement learning.

Original: In this paper, we specifically consider an agent that receives a single stream of observations from non-stationary clusters, with no ability to revisit past observations, but must nonetheless make progress towards grouping together “similar” observations.
Revised: In this paper, we specifically consider an agent who receives a single stream of observations from non-stationary clusters, with no ability to revisit past observations, but must nonetheless identify the clusters and assign observations to them.

Original: In this data regime, the number of clusters (i.e. the number of mixture components) is unknown and  unbounded, and so the agent must use a clustering algorithm capable of growing in representational capacity as more observations are seen.
Revised: In this data regime, the number of clusters (i.e. the number of mixture components) is unknown and theoretically could be unbounded, and so the agent must use a clustering algorithm capable of growing in representational capacity as more observations are encountered.

Original: In thisworkshop paper, we define a novel distribution over partitions of a set that we call the Dynamical Chinese Restaurant Process (Dynamical CRP), due to its relationship with theclassic Chinese 
Revised: In this paper, we define a novel distribution over partitions of a set that we call the Dynamical Chinese Restaurant Process (Dynamical CRP), due to its relationship with the Chinese Restaurant

Original: Restaurant Process (CRP).
Revised: Process (CRP) (Ferguson, 1973; Blackwell & MacQueen, 1973; Antoniak, 1974).

Original: We then show how the Dynamical CRP can be used as a prior over cluster assignments in a way that yields an efficient streaming inference algorithm designed for non-stationary data.
Revised: We then show how the Dynamical CRP can be used as a prior over cluster assignments in a manner that yields an efficient streaming clustering algorithm designed for non-stationary data.

Original: Finally, we conclude with preliminary experiments showing its applicability.
Revised: Starting with synthetic Gaussian and non-Gaussian data, and moving to more sophisticated real data based on simultaneous localization and mapping (SLAM) , we show that streaming inference using the Dynamical CRP achieves comparable or better performance than many common baselines, especially when the data is non-stationary.

Original:  B ACKGROUND 2. N OTATION
Revised:  B ACKGROUND 2. N OTATION

Original: We consider a single time series of D -dimensional observable variables o 1: N ( o n ∈ R D ) occurring at known times t 1: N , each corresponding to some latent cluster assignment variables c 1: N (i.e. c n ∈ { 1 , 2 , ... } ), where · 1: N denotes the sequence ( · 1 , · 2 , ..., · N ) .
Revised: We consider a single time series of D -dimensional observable variables o 1: N ( o n ∈ R D ) occurring at known times t 1: N and corresponding to some latent cluster assignment variables c 1: N (i.e. c n ∈ { 1 , 2 , ... } ), where · 1: N denotes the sequence ( · 1 , · 2 , ..., · N ) .

Original: Our goal is to infer the latent cluster assignments c 1: N .
Revised: Our goal is to infer the latent cluster assignments c 1: N .

Original: Each cluster may have corresponding variables { ϕ c } Cc =1 (e.g., per-cluster means and covariances) that we might also wish to infer.
Revised: Each cluster may have corresponding variables { ϕ c } C c =1 (e.g., per-cluster means and covariances) that we might also wish to infer.

Original: In the nonstationary setting, the clusters may change over time in a manner that we shall specify.
Revised: In the non-stationary setting, the clusters may change over time in a manner that we shall specify.

Original: 2. I NFINITE C LUSTERING VIA THE C HINESE R ESTAURANT P ROCESS
Revised: 2. I NFINITE C LUSTERING VIA THE C HINESE R ESTAURANT P ROCESS

Original: In the non-stationary streaming data regime, the number of clusters is unknown and unbounded.
Revised: In the non-stationary streaming data regime, the number of clusters is unknown and unbounded.

Original: Consequently, a useful clustering algorithm should be capable of (a) adding clusters as necessitated by the data, (b) generating predictions of future likely clusters, and (c) changing learnt representations of clusters over time.
Revised: Consequently, a useful clustering algorithm should be capable of (a) adding clusters as necessitated by the data, (b) generating predictions of future likely clusters, and (c) changing learnt representations of clusters over time.

Original: To meet the first two desiderata, many clustering algorithms use the
Revised: To meet the first two desiderata, many clustering algorithms use the

Original: Chinese Restaurant Process (CRP) or its related Dirichlet Process (Ferguson, 1973; Antoniak, 1974;
Revised: Chinese Restaurant Process (CRP) or its related Dirichlet Process (Ferguson, 1973; Antoniak, 1974;

Original: Neal, 2000; Blei & Jordan, 2006; Kulis & Jordan, 2012).
Revised: Neal, 2000; Blei & Jordan, 2006; Kulis & Jordan, 2012).

Original: The CRP is a single-parameter ( α > 0 ) stochastic process that defines a discrete distribution over partitions of a set, making it an appropriate prior for cluster assignments.
Revised: The CRP is a single-parameter ( α > 0 ) stochastic process that defines a discrete distribution over partitions of a set, making it an applicable prior for cluster assignments.

Original: The name CRP arises from a story of a sequence of customers (observations) arriving at a restaurant with an infinite number of tables (clusters), each table with infinite capacity.
Revised: The name CRP arises from a story of a sequence of customers (observations) arriving at a restaurant with an infinite number of tables (clusters), each table with infinite capacity.

Original: The first customer c 1 sits at the first table, and each subsequent customer c n sits either at an unoccupied table with probability proportional to α or joins an occupied table with probability proportional to the number of preceding customers at that table.
Revised: The first customer c 1 sits at the first table, and each subsequent customer c n sits either at an unoccupied table with probability proportional to α or joins an occupied table with probability proportional to the number of preceding customers at that table.

Original: Denoting the number of non-empty tables after the first n customers C n def = max( c 1 , ..., c n ) , CRP( α ) defines a conditional distribution for the n th customer c n given the preceding customers c <n :
Revised: Denoting the number of non-empty tables after the first n customers C n def = max( c 1 , ..., c n ) , CRP( α ) defines a conditional distribution for the n th customer c n given the preceding customers c <n :

Original: [Equation]
Revised: [Equation]

Original: Recent work showed that the CRP isuseful for task-free continual learning (Lee et al., 2020).
Revised: An example application of the CRP is task-free continual learning (Lee et al., 2020).

Original: However, the CRP is ill-suited to streaming data because the CRP’s conditional form requires knowing the entire history of cluster assignments. Schaeffer et al.
Revised: However, the CRP is ill-suited to streaming data because the CRP’s conditional form requires knowing the entire history of cluster assignments; Schaeffer et al.

Original: (2021) showedthat the CRP can be adapted for streaming data by rewriting the CRP in a recursive form:
Revised: (2021) showed the CRP can be adapted for streaming data by rewriting the CRP in a recursive form:

Original: [Equation]
Revised: [Equation]

Original: The two-part intuition is that (i) if many observations come from cluster c , then the next observation is also likely to come from cluster c , and (ii) the probability of more clusters should grow with the number of observations, giving the CRP the capacity to create an “infinite” number of clusters.
Revised: The two-part intuition is that (i) if many observations come from cluster c , then the next observation is also likely to come from cluster c , and (ii) the probability of more clusters should grow with the number of observations, giving the CRP the capacity to create an “infinite” number of clusters.

Original: 2. N ON - STATIONARY V ARIANTS OF THE C HINESE R ESTAURANT P ROCESS
Revised: 2. N ON - STATIONARY V ARIANTS OF THE C HINESE R ESTAURANT P ROCESS

Original: Although the CRP is widely used, the CRP has two properties which are inappropriate for nonstationary data.
Revised: Although the CRP is widely used, the CRP has two properties which are inappropriate for nonstationary data.

Original: First, the CRP is exchangeable, meaning permuting the order of the data does not affect the probability of the resulting partition.
Revised: First, the CRP is exchangeable, meaning permuting the order of the data does not affect the probability of the resulting partition.

Original: Second, the CRP is consistent, meaning marginalizing out any observation is the same as if the observation never existed.
Revised: Second, the CRP is consistent, meaning marginalizing out any observation is the same as if the observation never existed.

Original: To handle non-stationary data, Zhu et al.
Revised: To handle non-stationary data, Zhu et al.

Original: (2005) defined the time-sensitive CRP (tsCRP) by introducing exponential decay: 
Revised: (2005) defined the time-sensitive CRP (tsCRP) by introducing exponential decay: p tsCRP [Equation] n = [Equation]

Revised: ∝

Original: [Equation]
Revised: [Equation]

Original: otherwise(3)
Revised: otherwise

Original:  Blei & Frazier (2011) later defined the distance-dependent Chinese Restaurant Process (ddCRP), which assigns customers to other customers in a (possibly cyclic) directed graph.
Revised: [Equation] Blei & Frazier (2011) later defined the distance-dependent Chinese Restaurant Process (ddCRP), which assigns customers to other customers in a (possibly cyclic) directed graph.

Original: Whileextremely flexible, the ddCRP is impractical for streaming inference because(i) observations can be assigned to future observations, and (ii) the time and space complexity must be quadratic in the number of observations because the pairwise relationships have no structure and 
Revised: While flexible, the ddCRP is impractical for streaming inference because observations can be assigned to future observations, and time/space complexities must be quadratic in the number of observations because the pairwise relationships have no structure and thus must all be remembered.

Original:  M ETHODS 3. D ESIDERATA
Revised:  M ETHODS 3. D ESIDERATA

Original: Our goal is to define an efficient streaming inference algorithm for infinite non-stationary clustering.
Revised: Our goal is to define an efficient streaming inference algorithm for infinite non-stationary clustering.

Original: To do this, we define a novel stochastic process over partitions of a set called the Dynamical CRP to use as a prior over cluster assignments.
Revised: To do this, we define a novel stochastic process over partitions of a set called the Dynamical CRP to use as a prior over cluster assignments.

Original: The Dynamical CRP is designed with the following goals:
Revised: The Dynamical CRP is designed with the following goals:

Original: • Like the CRP, the Dynamical CRP can create “infinite” clusters (albeit upper bounded by the number of observations) and can generate predictions of the next likely clusters • Unlike the CRP, the Dynamical CRP does not assume the observations are  exchangeable or consistent, meaning the Dynamical CRP can model non-stationary data.
Revised: • Like the CRP, the Dynamical CRP can create “infinite” clusters (albeit upper bounded by the number of observations) and can generate predictions of the next likely clusters. • Unlike the CRP, the Dynamical CRP does not assume the observations are i.i.d., exchangeable or consistent, meaning the Dynamical CRP can model non-stationary data.

Original: • Unlike the tsCRP, the Dynamical CRP does not restrict the influence of observation times to exponential decay and can therefore capture a richer class of temporal relationships.
Revised: • Unlike the tsCRP, the Dynamical CRP does not restrict the influence of observation times to exponential decay and can therefore capture a richer class of temporal relationships.

Original: • Unlike the ddCRP, the Dynamical CRP admits an efficient streaming inference algorithm, which is critical for practical use by agents with finite memory.
Revised: • Unlike the ddCRP, the Dynamical CRP admits an efficient streaming inference algorithm, which is critical for practical use by agents with finite memory.

Original: The Dynamical CRP thus sits in a “Goldilocks” zone: more powerful than the CRP or tsCRP, but less powerful than the ddCRP so as to still permit efficient streaming inference.
Revised: The Dynamical CRP thus sits in a “Goldilocks” zone: more powerful than the CRP or tsCRP, but less powerful than the ddCRP so as to still permit efficient streaming inference.

Original: 3. H IGH L EVEL I DEA
Revised: 3. H IGH L EVEL I DEA

Original: The heart of the CRP are the “table occupancies” N c ( t ) def = (cid:80) n ′ ≤ n I ( c n = c )
Revised: The heart of the CRP is the “table occupancies” N c ( t ) def = (cid:80) n ′ ≤ n I ( c n = c )

Original: I ( t n  ≤ t ) , which are the sufficient statistics of the stochastic process.
Revised: I ( t n ′ ≤ t ) , which are the sufficient statistics of the stochastic process.

Original: The Dynamical CRP embeds those table occupancies in a dynamical system to evolve endogenously.
Revised: The Dynamical CRP embeds those table occupancies in a dynamical system to evolve endogenously.

Original: By choosing or learning dynamics appropriate for a particular task, the Dynamical CRP gains rich time-dependent priors for cluster assignments.
Revised: By choosing or learning dynamics appropriate for a particular task, the Dynamical CRP gains rich time-dependent priors for cluster assignments.

Original: 3. D EFINITION
Revised: 3. D EFINITION

Original: Let H be a Hilbert space and ˜ N
Revised: Let H be a Hilbert space and ˜ N

Original: ( t ) ∈
Revised: ( t ) ∈

Original: H contain both the “psuedo” table occupancies N c ( t ) and any desired higher-order temporal derivatives.
Revised: H contain both the “pseudo” table occupancies N c ( t ) and any desired higher-order temporal derivatives.

Original: Fix a linear dynamical system ℓ : ˜ N → ˜ N and increment the c n -th table N c n at time t n by 1 .
Revised: Fix a linear dynamical system ℓ : ˜ N → ˜ N and increment the c n -th table N c n at time t n by 1 .

Original: As before, define C n def = max( c 1 , ..., c n ) .
Revised: As before, define C n def = max( c 1 , ..., c n ) .

Original: The Dynamical CRP, denoted D - CRP ( ℓ, α ) , is defined as the conditional distribution:
Revised: The Dynamical CRP, denoted D - CRP ( ℓ, α ) , is defined as the conditional distribution:

Original: [Equation]
Revised: [Equation]

Original: Like the CRP, each customer increments a table’s occupancy count, but the tables’ occupancies can now change endogenously.
Revised: Like the CRP, each customer increments a table’s occupancy count, but the tables’ occupancies can now change endogenously.

Original: We next show the flexibility that the Dynamical CRP provides.
Revised: We next show the flexibility that the Dynamical CRP provides.

Original: 3. E XAMPLES
Revised: 3. E XAMPLES

Original: Step Clustering : If we define the dynamics as a perfect integrator ℓ ( ˜ N ) def = ∂ t ˜ N ( t ) with initial conditions N c (0) 
Revised: Stationary Dynamics : Define ℓ ( ˜ N ) def = ∂ t ˜ N ( t ) with initial conditions N c (0) = 0 .

Original: = 0 , the Dynamical CRP  simplifies to the CRP (Fig. 1 Time Function: Θ(∆) ).
Revised: Then the Dynamical CRP assumes the data is stationary and simplifies to the CRP (Fig. 1, Time Function: Θ(∆) ).

Original: Exponential Clustering : If we define the dynamics as a leaky integrator ℓ ( ˜ N ) def = ∂ t ˜ N ( t ) + ˜ N ( t ) , the relevance of previous customers (observations) decays exponentially with elapsed time and the Dynamical CRP simplifies to the time-sensitive CRP (Fig. 1 Time Function: exp( − ∆) ).
Revised: Exponential Dynamics : Define ℓ ( ˜ N ) def = τ∂ t ˜ N ( t )+ ˜ N ( t ) . Then the relevance of previous customers (observations) decays exponentially with elapsed time and the Dynamical CRP simplifies to the time-sensitive CRP (Fig. 1, Time Function: exp( − ∆) ).

Original: Oscillatory Clustering : Suppose we want cluster assignments to be periodic on a particular timescale.
Revised: Oscillatory Dynamics : Suppose we want cluster assignments to be periodic on a particular timescale.

Original: For instance, dawn and dusk have visually similar light, but crepuscular animals need to distinguish them.
Revised: For instance, dawn and dusk have visually similar light, but crepuscular animals need to distinguish them.

Original: Similarly, fall and spring have similar temperatures, but migratory and hibernating/aestivating animals need to distinguish them.
Revised: Similarly, fall and spring have similar temperatures, but migratory and hibernating/aestivating animals need to distinguish them.

Original: By defining the dynamics as a linear second order differential equation ℓ ( ˜ N ) def = ∂ 2 t ˜ N ( t )+ ∂ t ˜ N 
Revised: By defining the dynamics as a linear second order differential equation ℓ ( ˜ N ) def = ∂ 2 t ˜ N ( t ) + ˜ N ( t ) , the Dynamical CRP creates oscillatory table assignments (Fig. 1, Time Function: cos(∆) ).

Original: t )
Revised: Hyperbolic Dynamics : Hyperbolic discounting is commonly used in reinforcement learning and observed across species including humans, monkeys, and rats (Sozou, 1998; Fedus et al., 2019).

Original: + ˜ N ( t ) , the Dynamical CRP creates oscillatory table assignments (Fig. 1 Time Function: cos(∆) ).
Revised: The Dynamical CRP also enables hyperbolic clustering (Fig. 1, Time Function: 1 / (1 + ∆) ).

Original: 3. G ENERATIVE M ODEL
Revised: 3. G ENERATIVE M ODEL

Original: We now define the generative model for the streaming data o 1: N , observed at known times t 1: N , using the Dynamical CRP as a prior over cluster assignments c 1: N :
Revised: We now define the generative model for the streaming data o 1: N , observed at known times t 1: N , using the Dynamical CRP as a prior over cluster assignments c 1: N :

Original: [Equation]
Revised: [Equation]

Original: 3. S TREAMING I NFERENCE
Revised: 3. S TREAMING I NFERENCE

Original: Our approach will be to first show that the Dynamical CRP can be expressed in a recursive form designed for streaming inference, similarly to the CRP in (Schaeffer et al., 2021), and then use this recursive form as inspiration for a variational familyto use for streamingvariational inference.
Revised: Our approach will be to first show that the Dynamical CRP can be expressed in a recursive form designed for streaming inference, similarly to the CRP, and then use this recursive form to define a variational family for streaming inference.

Original: 3.6. R ECURSIVE F ORM OF THE D YNAMICAL CRP
Revised: 3.6. R ECURSIVE F ORM OF THE D YNAMICAL CRP

Original: As with the CRP, the Dynamical CRP’s conditional distribution renders each cluster assignment c n dependent on the entire history of previous cluster assignments c <n .
Revised: As with the CRP, the Dynamical CRP’s conditional distribution renders each cluster assignment c n dependent on the entire history of previous cluster assignments c <n .

Original: This handicaps its applicability to streaming data.
Revised: This handicaps its applicability to streaming data.

Original: We overcome  by converting the conditional distribution to a marginal distribution by taking the average over all possible  sample paths. Omitting p ( c n = c | ℓ, α ) = E p ( c n | ℓ,α ) [ I ( c n = c )] =
Revised: We overcome this by converting the conditional distribution to a marginal distribution by taking the average over all possible histories of cluster assignments (termed sample paths in the stochastic processes literature).

Original: E p ( c

Original: [Equation]

Original: n | ℓ, [Equation] [Equation] p ( c n | c <n ,ℓ,α ) [
Revised: Omitting t 1: N for brevity,

Original: [Equation]

Original: n

Original: [Equation]
Revised: [Equation]

Original: c )]]

Original: = E p ( c

Original: [Equation]

Original: n | ℓ,

Original: [Equation]

Original: ) [ p ( c n = c | c <n , ℓ, α )]

Original: Substituting the Dynamical CRP’s conditional distribution and taking a first-order Taylor series approximation of the expectation yields:
Revised: Substituting the Dynamical CRP’s conditional distribution and taking a first-order Taylor series approximation of the expectation yields:

Original: [Equation]
Revised: [Equation]

Original: Abusing notation slightly, we can write N c ( t  ) =
Revised: Abusing notation slightly, we can write N c ( t n ) =

Original: (cid:80) n ′ : t n ′ <
Revised: (cid:80) n ′ <n ℓ ( I ( c n ′ =

Original: t ℓ ( I ( c n ′ = c ) , t n ′ , t  ) , where ℓ ( · , t n ′ , t  ) means advancing the dynamical system from time t n ′ to time t  .
Revised: c ) , t n ′ , t n ) , where ℓ ( · , t n ′ , t n ) means advancing the dynamical system from time t n ′ to time t n .

Original: Because both the dynamics ℓ and the expectation are linear operators, the two commute and the expectation can be pulled inside:
Revised: Because both the dynamics ℓ and the expectation are linear operators, the two commute and the expectation can be pulled inside:

Original: [Equation]
Revised: [Equation]

Original: Together, this yields the following recursion:
Revised: Together, this yields the recursive form of the Dynamical CRP:

Original: [Equation]
Revised: [Equation]

Original: The Dynamical CRP’s recursive form (Eqn. 6) has similar intuition to the CRP’s recursive form (Eqn. 2): previous cluster assignments influence the current cluster assignment and clusters can appear with new observations.
Revised: The Dynamical CRP’s recursive form (Eqn. 6) has similar intuition to the CRP’s recursive form (Eqn. 2): previous cluster assignments influence the current cluster assignment and clusters can appear with new observations.

Original: The key modification is that previous probability masses can now change over time.
Revised: The key modification is that previous probability masses can now change over time.

Original: We confirm the correctness of Eqn. 6 by comparing the analytical expression to 5000 Monte Carlo samples drawn from the Dynamical CRP’s conditional distribution over α ∈ { 1 . 1 , 10 . 78 , 15 . 37 , 30 . 91 } and with step, exponential, sinusoidal and hyperbolic dynamics (Fig. 1); visually, the analytical and Monte Carlo plots display excellent agreement.
Revised: We confirm the correctness of Eqn. 6 by comparing the analytical expression to 5000 Monte Carlo samples drawn from the Dynamical CRP’s conditional distribution over α ∈ { 1 . 1 , 10 . 78 , 15 . 37 , 30 . 91 } and with step, exponential, sinusoidal, and hyperbolic dynamics (Fig. 1); visually, the analytical and Monte Carlo plots display excellent agreement.

Original: Quantitatively, the mean squared error between the analytical expression for all p ( c n | ℓ, α ) and the Monte Carlo estimates falls approximately as a power law in the number of Monte Carlo samples (Fig. 2) for all α values.
Revised: Quantitatively, the mean squared error between the analytical expression for all p ( c n | ℓ, α ) and the Monte Carlo estimates falls approximately as a power law in the number of Monte Carlo samples (Fig. 2) for all α values.

Original: This supports our claim that  recursive form of the Dynamical CRP is exact for the stochastic process.
Revised: This supports our claim that the recursive form of the Dynamical CRP is highly accurate.

Original: 3.6. S TREAMING I NFERENCE VIA R ECURSIVE F ORM OF D YNAMICAL CRP
Revised: 3.6. S TREAMING I NFERENCE VIA R ECURSIVE F ORM OF D YNAMICAL CRP

Original: To perform streaming inference, we start by considering the streaming evidence lower bound:
Revised: To perform streaming inference [Equation] we start by considering the streaming evidence lower bound:

Original: [Equation]
Revised: [Equation]

Original: with variational parameters θ n .
Revised: with variational parameters θ n .

Original: However, computing this ELBO is tricky because the filtering prior p ( c n , { ϕ }| o <n ) is unknown, so we replace the filtering prior with an approximate filtering prior:
Revised: However, computing this evidence lower bound is tricky because the filtering prior p ( c n , { ϕ }| o <n ) is unknown. Using the recursive form of the Dynamical CRP as inspiration [Equation] we replace the filtering prior with an approximate filtering prior:

Original: [Equation]
Revised: [Equation]

Original: Substituting the approximate filtering prior yields an approximate filtering evidence lower bound that we maximize:
Revised: Substituting the approximate filtering prior yields an approximate filtering evidence lower bound that we maximize:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: Following previous work (Kulis & Jordan, 2012), we started with synthetic mixtures of Gaussians. We generated datasets of 1000 observations by placing a Gaussian prior on the cluster means p ( ϕ ) = N (0 , ρ 2 I ) and sweeping over dynamics, alpha, signal-to-noise ratios, and number of dimensions to construct 3600 datasets total.

Original: E XPERIMENTAL R ESULTS 4. R OOM C LUSTERING FOR S IMULTANEOUS L OCALIZATION AND M APPING
Revised: We compared Dynamical CRP against seven baseline inference algorithms; three are streaming and four are not. The non-streaming algorithms have unfettered access to all observations and therefore serve as upper bounds on performance; any comparison against these non-streaming baselines maximally disfavors our method. The baselines are: • Collapsed Gibbs Sampling (non-streaming) (Neal, 2000).

Revised: • Variational Bayes Dirichlet-Process Gaussian Mixture Model (non-streaming)

Revised: (Blei & Jordan, 2006), implemented in scikit-learn (Pedregosa et al., 2011).

Revised: • K-Means (both streaming and non-streaming variants) (MacQueen, 1967; Lloyd, 1982).

Revised: For both variants, K-Means is given the ground-truth number of clusters.

Revised: • DP-Means (both streaming and non-streaming variants) (Kulis & Jordan, 2012; Broderick et al., 2013).

Revised: • Recursive-CRP

Revised: (streaming) (Schaeffer et al., 2021).

Revised: The baseline algorithms are all designed for stationary data, so we started our comparison with stationary dynamics, i.e. ℓ ( ˜ N ) =

Revised: ∂ t ˜ N ( t ) , but we also considered other dynamics (exponential, oscillatory, hyperbolic).

Revised: We measured the performance of each algorithm by the (normalized) mutual information between the inferred cluster assignments and the true cluster assignments, implemented in scikit-learn (Pedregosa et al., 2011).

Revised: On stationary data, we found that Dynamical CRP was competitive on stationary data (Fig 3A), but excelled on non-stationary data (Fig 3BCD).

Revised: We additionally plotted the number of clusters inferred by each algorithm.

Revised: We found that D-CRP was often well within the correct order of magnitude, growing appropriately with the concentration hyperparameter α (Fig. 4).

Revised: To explore how clusters are created as observations are received, we visualized when the Dynamical CRP creates clusters by plotting the ratio of the number of inferred clusters to the total number of true clusters as a function of the number of observations, dividing by the total number of true clusters in that set of observations.

Revised: We found that Dynamical CRP creates clusters over time, as necessitated by the data (Fig. 5).

Revised: We also investigated how Dynamical CRP performs under different signal-to-noise (defined as the ratio of means covariance prefactor ρ to likelihood covariance prefactor σ ) regimes; interestingly, we found that while some algorithms do not display better performance with increasing signal-to-noise,

Revised: Dynamical CRP does display better performance with higher SNR (Fig. 6).

Revised: 4. S YNTHETIC M IXTURE OF VON M ISES -F ISHER

Revised: To demonstrate that the Dynamical CRP is not limited to Gaussian mixture models in Euclidean space, we turned to von Mises-Fisher mixture models on the surface of hyperspheres.

Revised: We made this particular choice because one future line of work we are excited by involves combining deep learning with Bayesian nonparametrics for lifelong learning, and recent advances in self-supervised representation learning constrain deep neural network representations to the surface of hyperspheres (Chen et al., 2020;

Revised: Grill et al., 2020; Caron et al., 2021).

Revised: As with the mixture of Gaussians, we generated datasets of 1000 observations from the generative model, with a uniform prior on the cluster directions p ( ϕ ) =

Revised: VMF ( κ = 0) , by sweeping over dynamics, alpha, signal-to-noise ratios, and ambient dimension to construct 3600 datasets total.

Revised: Most previous baselines were designed for Gaussian likelihoods, meaning only the Recursive-CRP could be used.

Revised: We again plotted the number of clusters inferred by each algorithm.

Revised: We found that D-CRP often outperformed R-CRP

Revised: (Fig. 7) and was often well within the correct order of magnitude, growing appropriately with the concentration hyperparameter α (Fig.

Revised: 4. R OOM C LUSTERING FOR S IMULTANEOUS L OCALIZATION AND M APPING

Original: Clustering is useful not just in its own right, but also as a sub-task for other tasks.
Revised: Clustering is useful not just in its own right, but also as a sub-task for other tasks.

Original: For instance, an agent may wish to clustersensory observations into states for use in planning or reinforcement learning.
Revised: For instance, an agent may wish to cluster observations into states for use in planning or reinforcement learning.

Original: As a demonstration, we turn to the well-studied problem of simultaneous localization and mapping (SLAM)
Revised: As a demonstration, we turn to the domain of simultaneous localization and mapping (SLAM)

Original: (Rosen et al., 2021), in which an agent must both learn a map of its environment as well as its location within that environment.
Revised: (Rosen et al., 2021), in which an agent must both learn a map of its environment as well as its location within that environment.

Original: One common approach is to learn hierarchically (Fairfield et al., 2010; Klukas et al., 2022) by clustering sensory observations into rooms that can then be used to efficiently plan by planning at the level of rooms instead of at the level of granular positions.
Revised: One common approach is to learn hierarchically (Fairfield et al., 2010; Klukas et al., 2022) by clustering sensory observations into rooms that can then be used to efficiently plan.

Original: We procedurally generated environments with multiple rooms, each containing a variable number of(binary) sensory observations commonly called “landmarks” in the SLAM literature, and then applied D-CRP  to a single trajectory through each novel environment.
Revised: We procedurally generated environments with multiple rooms, each containing a variable number of sensory observations (“landmarks” in the SLAM literature), and then applied D-CRP with exponential dynamics to a single trajectory through each novel environment.

Original:  We implemented inference in this setting using a product-of-independent-Bernoullis likelihood for expressing whether each landmark is visible from a given position;  the model does not take into account position or velocity information.
Revised: At each position along the trajectory, a landmark is either visible or not, determined by a limited range of view. We used a product-of-Bernoullis likelihood for expressing whether each landmark is visible from a given position; for simplicity, the likelihood does not take into account position or velocity.

Original: We found the D-CRP with exponentially decaying dynamics excels at this task (Fig 3).
Revised: We found the D-CRP with exponentially decaying dynamics excels at this task.

Original: As D-CRP takes a single trajectory through each novel environment, D-CRP aggregates sensory landmarks (black diamonds) into visually-distinguishable unique clusters (rooms) and visually-indistinguishable non-unique clusters (hallways, orange).
Revised: As D-CRP takes a single trajectory through each novel environment, D-CRP aggregates sensory landmarks (black diamonds) into visually-distinguishable unique clusters (rooms) and visually-indistinguishable non-unique clusters (hallways, orange) (Fig 9).

Revised: The D-CRP excels at this task when equipped with exponentially decaying dynamics, because the dynamics impose an inductive bias that trajectories are temporally and spatially smooth, meaning the

Revised: D-CRP has a strong prior towards allocating two sequential observations to the same cluster even if the two sensory observations differ significantly.

Original: These rooms could then serve as a high-level representation for hierarchical spatial navigation.
Revised: These rooms could then serve as a high-level representation for hierarchical spatial navigation.

Original: The D-CRP’s excels because its exponentially decaying dynamics gives the D-CRP prior knowledge that trajectories are (typically) temporally and spatially smooth, meaning the D-CRP has a strong prior towards allocating two sequential observations to the same cluster even if the two sensory observations differ significantly. landmarks (black diamonds) encountered along a single trajectory. The Dynamical CRP aggregates visually-distinguishable rooms into distinct clusters and visually-identical hallways into the same cluster (orange). 4. S YNTHETIC M IXTURE OF M ULTIVARIATE G AUSSIANS  D ISTRIBUTIONS
Revised:  D ISCUSSION

Original: Following previous work e.g. Kulis & Jordan (2012), we tested our inference algorithm on synthetic mixture of Gaussians data.
Revised: In this paper, we attack unsupervised learning on streaming non-stationary data in the specific setting of mixture modeling.

Original: We sampled 1000 observations from Gaussian mixture models with a  Dynamical CRP  prior over cluster assignments, assuming known cluster variances, sweeping dynamics, alpha, signal-to-noise ratio and ambient dimension. We found that our streaming  inference algorithm excelled at recovering the cluster assignments, as measured by normalized mutual information between the inferred cluster assignments and  the true cluster assignments (Fig 4).
Revised: We propose a novel stochastic process that defines a non-exchangable distribution over partitions of a set, that we termed the Dynamical Chinese Restaurant Process. We show that the Dynamical CRP provides a bespoke non-stationary prior over cluster assignments and is amenable to an efficient streaming variational inference algorithm. We then demonstrate on both synthetic and real data, with Gaussian and non-Gaussian likelihoods, that the Dynamical CRP provides a powerful clustering algorithm for non-stationary streaming data.

Original: We also tested how well our streaming inference algorithm performs at recovering the cluster assignments as a function of signal-to-noise, again considering four different types of dynamics and measuring performance with the normalized mutual information between true cluster assignments and inferred cluster assignments. We generally found that performance increases with higher signalto-noise (Fig. 5), although not always, meriting further investigation.  D ISCUSSION
Revised: A C OORDINATE A SCENT V ARIATIONAL I NFERENCE P ARAMETER U PDATES

Original: Ultimately, we ran out of time for  the experimental results we wanted, but we intend to continue working on this and will hopefully have better results come April!
Revised: This section contains derivations for Coordinate Ascent Variational Inference (CAVI). Specifically, the following derivations show how to update variational parameters in different with various likelihoods.

Original: A C OORDINATE A SCENT V ARIATIONAL I NFERENCE P ARAMETER U PDATES A. C LOSED -F ORM E XPRESSION FOR V ARIATIONAL P ARAMETERS IN THE E XPONENTIAL F AMILY
Revised: A. C LOSED -F ORM E XPRESSION FOR V ARIATIONAL P ARAMETERS IN THE E XPONENTIAL F AMILY

Original: In the following three subsections, we use the following fact Beal (2003);
Revised: In the following three subsections [Equation]

Original:  Wainwright & Jordan (2008): if a distribution p and its variational approximation q are both in the exponential family, then the optimal variational parameters ζ i that correspond to the variational distribution over variable W i are the solution to
Revised: we use the following fact from Beal (2003); Wainwright & Jordan (2008): if a distribution p and its variational approximation q are both in the exponential family, then the optimal variational parameters ζ i that correspond to the variational distribution over variable W i are the solution to

Original: [Equation]
Revised: [Equation]

Original: This means that when optimizing the parameters for one variable, we can replace all other variables with their expectations under the variational distribution and then solve for that one variable’s  parameters.
Revised: This means that when optimizing the parameters for one variable, we can replace all other variables with their expectations under the variational distribution and then solve for that one variable’s variational parameters.

Original: A. CAVI FOR M ULTIVARIATE G AUSSIAN L IKELIHOOD
Revised: A. CAVI FOR M ULTIVARIATE G AUSSIAN L IKELIHOOD

Original: Mean field family:
Revised: Mean field family:

Original: [Equation]
Revised: [Equation]

Original: where θ n def = { π nc } k ∪ { µ nc } k ∪ { Σ nc } k are our variational parameters for the n th observation.
Revised: where θ n def = { π nc } k ∪ { µ nc } k ∪ { Σ nc } k are our variational parameters for the n th observation.

Original: The mixture weights’ parameters π n will be determined by solving the following:
Revised: The mixture weights’ parameters π n will be determined by solving the following:

Original: [Equation]
Revised: [Equation]

Original: The left-hand side (LHS) is:
Revised: The left-hand side (LHS) is:

Original: [Equation]
Revised: [Equation]

Original: Dropping terms that don’t include c n , the right-hand side (RHS) contains two relevant terms:
Revised: Dropping terms that don’t include c n , the right-hand side (RHS) contains two relevant terms:

Original: [Equation]
Revised: [Equation]

Original: The first term is determined by the Dynamical CRP prior:
Revised: The first term is determined by the Dynamical CRP prior:

Original: [Equation]
Revised: [Equation]

Original: The second term is given by:
Revised: The second term is given by:

Original: [Equation]
Revised: [Equation]

Original: Comparing the simplified left-hand and right-hand sides

Original: [Equation]

Original: and solving for the variational parameter of the probability of the l th cluster π nl :
Revised: of the probability of the l th cluster π nl :

Original: [Equation]
Revised: [Equation]

Original: For the l th cluster centroid’s variational parameters, we want to solve:
Revised: For the l th cluster centroid’s variational parameters, we want to solve:

Original: [Equation]
Revised: [Equation]

Original: The LHS is:
Revised: The LHS is:

Original: [Equation]
Revised: [Equation]

Original: The RHS is:
Revised: The RHS is:

Original: [Equation]
Revised: [Equation]

Original: where the first RHS term is:
Revised: where the first RHS term is:

Original: [Equation]
Revised: [Equation]

Original: and the second RHS term is:
Revised: and the second RHS term is:

Original: [Equation]
Revised: [Equation]

Original: Setting the LHS and RHS equal and isolating terms quadratic in ϕ nl allows us to solve for the variational covariance parameter:
Revised: Setting the LHS and RHS equal and isolating terms quadratic in ϕ nl allows us to solve for the variational covariance parameter:

Original: [Equation]
Revised: [Equation]

Original: Isolating terms linear in ϕ nl allows us to solve for the variational mean parameter:
Revised: Isolating terms linear in ϕ nl allows us to solve for the variational mean parameter:

Original: [Equation]
Revised: [Equation]

Original: A. CAVI FOR (P RODUCT OF ) B ERNOULLI L IKELIHOODS
Revised: A. CAVI FOR (P RODUCT OF ) B ERNOULLI L IKELIHOODS

Original: Mean-field family:
Revised: Mean-field family:

Original: [Equation]
Revised: [Equation]

Original: where θ n def = { π nc } c ∪ { γ ncl } ∪ { κ ncl } are our variational parameters for the n th observation.
Revised: where θ n def = { π nc } c ∪ { γ ncl } ∪ { κ ncl } are our variational parameters for the n th observation.

Original: The mixture weights’ parameters π n will be determined by solving the following:
Revised: The mixture weights’ parameters π n will be determined by solving the following:

Original: [Equation]
Revised: [Equation]

Original: The left-hand side (LHS) is:
Revised: The left-hand side (LHS) is:

Original: [Equation]
Revised: [Equation]

Original: Dropping terms that don’t include c n , the right-hand side (RHS) contains two relevant terms:
Revised: Dropping terms that don’t include c n , the right-hand side (RHS) contains two relevant terms:

Original: [Equation]
Revised: [Equation]

Original: The first term is determined by the Dynamical CRP prior:
Revised: The first term is determined by the Dynamical CRP prior:

Original: [Equation]
Revised: [Equation]

Original: The second term is given by:
Revised: The second term is given by:

Original: [Equation]
Revised: [Equation]

Original: where ψ ( x ) def = ddx log Γ( x ) is the digamma function.
Revised: where ψ ( x ) def = ddx log Γ( x ) is the digamma function.

Original: Comparing the simplified left-hand and righthand sides, and solving for the variational parameter of the probability of the l th cluster π nl :
Revised: Comparing the simplified left-hand and righthand sides, and solving for the variational parameter of the probability of the l th cluster π nl :

Original: [Equation]
Revised: [Equation]

Original: For the c -th cluster’s variational parameters [Equation] we want to solve:
Revised: For the c -th cluster’s variational parameters, we want to solve:

Original: [Equation]
Revised: [Equation]

Original: The LHS is:
Revised: The LHS is:

Original: [Equation]
Revised: [Equation]

Original: Dropping terms that don’t contain ϕ ncl , the RHS is:
Revised: Dropping terms that don’t contain ϕ ncl , the RHS is:

Original: [Equation]
Revised: [Equation]

Original: Grouping terms and setting equal:
Revised: Grouping terms and setting equal:

Original: [Equation]
Revised: [Equation]

Original: A. CAVI FOR VON -M ISES -F ISHER L IKELIHOOD
Revised: A. CAVI FOR VON -M ISES -F ISHER L IKELIHOOD

Original: Mean field family:
Revised: Mean field family:

Original: [Equation]
Revised: [Equation]

Original: where θ n def = { π nc } k ∪ { µ nc } k ∪ { κ nc } k are our variational parameters for the n th observation.
Revised: where θ n def = { π nc } k ∪ { µ nc } k ∪ { κ nc } k are our variational parameters for the n th observation.

Original: The mixture weights’ parameters π n will be determined by solving the following:
Revised: The mixture weights’ parameters π n will be determined by solving the following:

Original: [Equation]
Revised: [Equation]

Original: The left-hand side (LHS) is:
Revised: The left-hand side (LHS) is:

Original: [Equation]
Revised: [Equation]

Original: Dropping terms that don’t include c n , the right-hand side (RHS) contains two relevant terms:
Revised: Dropping terms that don’t include c n , the right-hand side (RHS) contains two relevant terms:

Original: [Equation]
Revised: [Equation]

Original: The first term is determined by the RNCRP prior:
Revised: The first term is determined by the RNCRP prior:

Original: [Equation]
Revised: [Equation]

Original: The second term is given by:
Revised: The second term is given by:

Original: [Equation]
Revised: [Equation]

Original: Comparing the simplified left-hand and right-hand sides, and solving for the variational parameter of the probability of the l th cluster π nl :
Revised: Comparing the simplified left-hand and right-hand sides, and solving for the variational parameter of the probability of the l th cluster π nl :

Original: [Equation]
Revised: [Equation]

Original: For the l th cluster centroid’s variational parameters, we want to solve:
Revised: For the l th cluster centroid’s variational parameters, we want to solve:

Original: [Equation]
Revised: [Equation]

Original: The LHS is:
Revised: The LHS is:

Original: [Equation]
Revised: [Equation]

Original: The RHS is:
Revised: The RHS is:

Original: [Equation]
Revised: [Equation]

Original: where the first RHS term is:
Revised: where the first RHS term is:

Original: [Equation]
Revised: [Equation]

Original: and the second RHS term is:
Revised: and the second RHS term is:

Original: [Equation]
Revised: [Equation]

Original: Setting the LHS and RHS equal:
Revised: Setting the LHS and RHS equal:

Original: [Equation]
Revised: [Equation]

Original: The two variational parameters are separately recoverable by setting µ nl equal to the unit direction of the right hand side RHS def = κ n − 1 ,l µ n − 1 ,l + 1 σ 2 o π nl o n and by setting κ nl equal to the magnitude of the right hand side:
Revised: The two variational parameters are separately recoverable by setting µ nl equal to the unit direction of the right hand side RHS def = κ n − 1 ,l µ n − 1 ,l + 1 σ 2 o π nl o n and by setting κ nl equal to the magnitude of the right hand side:

Original: [Equation]
Revised: [Equation]