Paper: Change Point Detection via Variational Time-Varying Hidden Markov Model (ICLR-2024)

Reviewer comment (Writing):
The current version contains a few typographical errors and some notational issues, which make it somewhat challenging to read.

Author response:
Q4: “The current version contains a few typographical errors and some notational issues, which make it somewhat challenging to read.”\
\
Thank you very much for the correction. We’re sorry about the typographical errors and the notational issues in the paper and have already corrected them. We have changed all $S_k$ into $\Lambda_k$ and replaced $K$ with $ \tilde K$ in Algorithm 2, for consistent usage of symbols. The $\theta_k$ in Section 4 is substituted with $\zeta_k$ to avoid confusion.

Reviewer comment (Experiments):
It seems that the results and details of the simulations corresponding to Section 4 (the semi-paramteric model) are not given in the paper. (In particular, I would have been curious to know how the author select the mapping $\phi$ in their simulations.)

Author response:
Experiment:\
\
We also compared the semi-parametric TV-HMM with DPHMM on a sequence with piecewise linear segments to show the improvement. In this experiment, the data are generated from the model $\beta_0+\beta_1*t+\epsilon$, $\epsilon$ follows a normal distribution with mean 0 and variance 1. The length of the sequence is 600 with 4 inequal distanced change points. The change point locations are $\tau=[ 80 , 230, 330, 480]$ indicating lengths of each segments are $[ 80,150,100,150,120]$. $\tilde{K}$ is 10. The $\beta_0$ and $\beta_1$ are shown in the following table. 

Model: $y(t)=\beta_0+\beta_1*t+\epsilon$,  $\epsilon \sim N(0,1)$\
Total sample size: $N=600$\
Change point location: $\tau=[ 80 , 230, 330, 480]$\
Initialized number of change point: $\tilde{K}=10$

|           | Segment 1 | Segment 2 |  Segment 3 |  Segment 4 | Segment 5 |
|----------:|:---------:|:---------:|:----------:|:----------:|:----------:|
| $\beta_0$ |     0     | $seg_1(80)$ |$ seg_2(150)$ | $seg_3(100)$ | $seg_4(150)$ |
| $\beta_1$ |    0.1    |    -0.3   |     0.3    |    -0.2    |    -0.2    |

 $seg_i(j)$ stands for the $j$-th element in $i$-th segment.\
\
The summarized average Rand Index and its standard deviation are shown in the following table. It can be observed that Semi-parametric TV-HMM outperforms the other compared methods, indicating that our proposed method can handle the change point detection problem in sequences with linear trends. Moreover, the standard deviation of the Randx Index measurement is 0.02512, demonstrating the stability of our method.
|            |   semi-TVHMM   |      DPHMM     | ECP3O       |           WBSLSW |       KCP      |   $D_m$-BOCD   |
|-----------:|:--------------:|:--------------:|----------------|:--------------:|:--------------:|:--------------:|
| Rand Index |      0.9533    |      0.7882    |      0.8580    |      0.2209    |      0.8199    |      0.7637    |
|     SD     |     0.02512    |     0.06822    |     0.00422    |     0.05569    |     0.00197    |     0.09845    |

Reviewer comment (Writing):
In assumption A1, I think in the Gaussian distribution $S_k$ should be $\Lambda_k$ (to be consistent with the notations used in Eq.(1)).

Author response:
Q4: “The current version contains a few typographical errors and some notational issues, which make it somewhat challenging to read.”\
\
Thank you very much for the correction. We’re sorry about the typographical errors and the notational issues in the paper and have already corrected them. We have changed all $S_k$ into $\Lambda_k$ and replaced $K$ with $ \tilde K$ in Algorithm 2, for consistent usage of symbols. The $\theta_k$ in Section 4 is substituted with $\zeta_k$ to avoid confusion.

Reviewer comment (Writing):
In Section 4, the use of the notation $\theta_k$ is confusing. Indeed, before section 4, we work in the full parametric case and $\theta_k=(u_k,\Lambda_k)$ (the parameter of the Gaussian distribution from which points are independently sampled from after the $k$-th change point). However, $\theta_k$ in Section 4 needs to be a vector in $\mathbb R^D$ (i.e. in the same space as the observations). Therefore, it might be good to use another notation or to stress at the beginning of Section 4 the properties of $\theta_k$.

Author response:
Q4: “The current version contains a few typographical errors and some notational issues, which make it somewhat challenging to read.”\
\
Thank you very much for the correction. We’re sorry about the typographical errors and the notational issues in the paper and have already corrected them. We have changed all $S_k$ into $\Lambda_k$ and replaced $K$ with $ \tilde K$ in Algorithm 2, for consistent usage of symbols. The $\theta_k$ in Section 4 is substituted with $\zeta_k$ to avoid confusion.

Reviewer comment (Reproducibility):
In Eq.(4), the parameters $\pi_{k,m,n}$ are still considered but the update from line 8 (Algo 1) is not presented anymore in Algo 2. I would be grateful if the authors could clarify this point.

Author response:
Q2: “In Eq.(4), the parameters \pi are still considered but the update from line 8 (Algo 1) is not presented anymore in Algo 2. I would be grateful if the authors could clarify this point. ”\
\
Indeed, the parameters $\pi$ still need to be considered in Algorithm 2, and we have added one more step for $\pi$ in the Algorithm.

Reviewer comment (Writing):
Below Algorithm 2, there should not be a final point: "Unlike Equation 2 in the parametric model, where $Q(\theta)$ must be derived using variational inference. Here, $Q(\theta)$ can be generally modeled using non-parametric density estimation...". Furthermore, using "must" suggests that no alternative can be considered. However, I would say that standard MCMC techniques could be used instead of variational inference. If the authors agree, I would suggest to reformulate the sentence. In the second sentence, it is also mention that $Q$ can be modeled using non parametric techniques, but the following sentence (and the presented algorithm) only deal with a parametric distribution (with parameter $\Phi$). It might be good to mention how the algorithm is changed when a non-parametric density estimation method is used.

Author response:
Q3: “Furthermore, using "must" suggests that no alternative can be considered. However, I would say that standard MCMC techniques could be used instead of variational inference. If the authors agree, I would suggest to reformulate the sentence. In the second sentence, it is also mention that \Q can be modelled using non parametric techniques, but the following sentence (and the presented algorithm) only deal with a parametric distribution (with parameter ). It might be good to mention how the algorithm is changed when a non-parametric density estimation method is used.”\
\
Thank you very much for your suggestion. Here we use "must" because $Q(\theta)$ with explicit form in Algorithm 1 is developed under the framework of Variational Inference. Although we can use MCMC to perform inference on parametric TV-HMM, the computational cost is higher. The main difference between the non-parametric and parametric TV-HMM is that we substitute the parametric likelihood measurement with the Maximum Mean Discrepancy(MMD). We first sample $\tilde K+1$  sequences from non-parametrically estimated $Q(\zeta)$. Each $Q(\zeta_k)$ represents a distinguished regime distribution of the sequence. Calculating the MMD between the sampled sequences and the original sequence, we can find the best-matching segment to determine the change point location. Although the parametric model assumption can help derive analytic solutions and enhance the convergence rate, the model may become compromised if the assumption is violated. e.g., the DPHMM method is not well performed on a sequence with piecewise linear segments.

Reviewer comment (Reproducibility):
With the current details, it would be difficult to implement the method and reproduce the results of the paper. An additional section with implementation details is suggested to ensure reproducibility.

Author response:
Thank you very much for your advice. We have summarized all the experiment settings in Appendix D. The input parameters such as the initialized number of change points $\tilde K$, the sample sequence $Y$, and how it is generated are all included. We also add an additional experiment about the piecewise linear sequence. We are delighted to release the code after entering the final revision. Thank you again for your understanding and support.

Reviewer comment (Experiments):
The experimental setup is limited. In particular, the simulation study includes equally spaced change points, which is quite simplistic.

Author response:
Thank you very much for your suggestion. We also test the semi-parametric TV-HMM on a sequence with piecewise linear segments to show the improvement. In this experiment, the data are generated from the model $\beta_0+\beta_1*t+\epsilon$, $\epsilon$ follows a normal distribution with mean 0 and variance 1. The length of the sequence is 600 with 4 inequal distanced change points. The change point locations are $\tau=[ 80 , 230, 330, 480]$ indicating lengths of each segments are $[ 80,150,100,150,120]$. $\tilde{K}$ is 10. The $\beta_0$ and $\beta_1$ are shown in the following table. 

Model: $y(t)=\beta_0+\beta_1*t+\epsilon$,  $\epsilon \sim N(0,1)$\
Total sample size: $N=600$\
Change point location: $\tau=[ 80 , 230, 330, 480]$\
Initialized number of change point: $\tilde{K}=10$

|           | Segment 1 | Segment 2 |  Segment 3 |  Segment 4 | Segment 5 |
|----------:|:---------:|:---------:|:----------:|:----------:|:----------:|
| $\beta_0$ |     0     | $seg_1(80)$ |$ seg_2(150)$ | $seg_3(100)$ | $seg_4(150)$ |
| $\beta_1$ |    0.1    |    -0.3   |     0.3    |    -0.2    |    -0.2    |

 $seg_i(j)$ stands for the $j$-th element in $i$-th segment.\
\
The summarized average Rand Index and its standard deviation are shown in the following table. It can be observed that Semi-parametric TV-HMM outperforms the other compared methods, indicating that our proposed method can handle the change point detection problem in sequences with linear trends. Moreover, the standard deviation of the Randx Index measurement is 0.02512, demonstrating the stability of our method.
|            |   semi-TVHMM   |      DPHMM     |     ECP3O    |      WBSLSW      |       KCP      |   $D_m$-BOCD   |
|-----------:|:--------------:|:--------------:|----------------|:--------------:|:--------------:|:--------------:|
| Rand Index |      0.9533    |      0.7882    |      0.8580    |      0.2209    |      0.8199    |      0.7637    |
|     SD     |     0.02512    |     0.06822    |     0.00422    |     0.05569    |     0.00197    |     0.09845    |

Reviewer comment (Evaluation):
There is no computational comparison between the methods, making it unclear how much one has to sacrifice in terms of speed to gain a little extra performance.

Author response:
The novelty is about extending the Hidden Markov Model. Since our proposed method is based on Variational Inference, the computational speed is indeed faster than previous DPHMM using MCMC sampling. The computational complexity for our proposed method is $\mathcal{O}(KN^2)$. If equipped with stochastic approximation, the complexity can be reduced to $\mathcal{O}(KS^2)$, and $S$ is the sub-sample size. The computational speed we can achieve for each step during iteration is approximately 1.73 seconds.

Reviewer comment (Evaluation):
There are somewhat marginal differences in the performance between the proposed approach and other methods (both when the proposed approach underperforms and overperforms). From the current evaluation, it is unclear in what scenarios it would be beneficial to use the proposed method.

Author response:
Thank you very much for your suggestion. We also test the semi-parametric TV-HMM on a sequence with piecewise linear segments to show the improvement. In this experiment, the data are generated from the model $\beta_0+\beta_1*t+\epsilon$, $\epsilon$ follows a normal distribution with mean 0 and variance 1. The length of the sequence is 600 with 4 inequal distanced change points. The change point locations are $\tau=[ 80 , 230, 330, 480]$ indicating lengths of each segments are $[ 80,150,100,150,120]$. $\tilde{K}$ is 10. The $\beta_0$ and $\beta_1$ are shown in the following table. 

Model: $y(t)=\beta_0+\beta_1*t+\epsilon$,  $\epsilon \sim N(0,1)$\
Total sample size: $N=600$\
Change point location: $\tau=[ 80 , 230, 330, 480]$\
Initialized number of change point: $\tilde{K}=10$

|           | Segment 1 | Segment 2 |  Segment 3 |  Segment 4 | Segment 5 |
|----------:|:---------:|:---------:|:----------:|:----------:|:----------:|
| $\beta_0$ |     0     | $seg_1(80)$ |$ seg_2(150)$ | $seg_3(100)$ | $seg_4(150)$ |
| $\beta_1$ |    0.1    |    -0.3   |     0.3    |    -0.2    |    -0.2    |

 $seg_i(j)$ stands for the $j$-th element in $i$-th segment.\
\
The summarized average Rand Index and its standard deviation are shown in the following table. It can be observed that Semi-parametric TV-HMM outperforms the other compared methods, indicating that our proposed method can handle the change point detection problem in sequences with linear trends. Moreover, the standard deviation of the Randx Index measurement is 0.02512, demonstrating the stability of our method.
|            |   semi-TVHMM   |      DPHMM     |     ECP3O    |      WBSLSW      |       KCP      |   $D_m$-BOCD   |
|-----------:|:--------------:|:--------------:|----------------|:--------------:|:--------------:|:--------------:|
| Rand Index |      0.9533    |      0.7882    |      0.8580    |      0.2209    |      0.8199    |      0.7637    |
|     SD     |     0.02512    |     0.06822    |     0.00422    |     0.05569    |     0.00197    |     0.09845    |

Reviewer comment (Evaluation):
Table 1 lacks standard deviations over the 100 runs.

Author response:
Thank you very much for pointing out these issues. We have added the standard deviations over 100 runs and included the required information. We also check the reference section to ensure each reference includes journal information or arxiv identifier. Missing labels in the figures are fixed and captions are adjusted.
|            |     Model 1     |     Model 2     |      Model 3     |
|:----------:|:---------------:|:---------------:|:----------------:|
|   WBSLSW   | 0.9068(0.08744) | 0.3596(0.22447) |  0.3849(0.23038) |
|    ECP30   | 0.9156(0.04346) | 0.9580(0.05754) |  0.9737(0.01215) |
| DPHMM      | 0.9637(0.02257) | 0.8727(0.03066) | 0.8869(0.02651)  |
| KCP        | 0.9501(0.05622) | 0.8436(0.01631) | 0.8836(0.04664)  |
| $D_m$-BOCD | 0.8123(0.11902)  | 0.8411(0.01894) | 0.8413(0.02208) |
| TV-HMM     | 0.9523(0.05174) | 0.9756(0.01339) | 0.9615(0.02004)  |

Reviewer comment (Presentation):
Some references do not include journal information or arXiv identifier.

Author response:
Thank you very much for pointing out these issues. We have added the standard deviations over 100 runs and included the required information. We also check the reference section to ensure each reference includes journal information or arxiv identifier. Missing labels in the figures are fixed and captions are adjusted.
|            |     Model 1     |     Model 2     |      Model 3     |
|:----------:|:---------------:|:---------------:|:----------------:|
|   WBSLSW   | 0.9068(0.08744) | 0.3596(0.22447) |  0.3849(0.23038) |
|    ECP30   | 0.9156(0.04346) | 0.9580(0.05754) |  0.9737(0.01215) |
| DPHMM      | 0.9637(0.02257) | 0.8727(0.03066) | 0.8869(0.02651)  |
| KCP        | 0.9501(0.05622) | 0.8436(0.01631) | 0.8836(0.04664)  |
| $D_m$-BOCD | 0.8123(0.11902)  | 0.8411(0.01894) | 0.8413(0.02208) |
| TV-HMM     | 0.9523(0.05174) | 0.9756(0.01339) | 0.9615(0.02004)  |

Reviewer comment (Presentation):
Figures are sometimes missing labels and captions of the figures/tables are not self-contained.

Author response:
Thank you very much for pointing out these issues. We have added the standard deviations over 100 runs and included the required information. We also check the reference section to ensure each reference includes journal information or arxiv identifier. Missing labels in the figures are fixed and captions are adjusted.
|            |     Model 1     |     Model 2     |      Model 3     |
|:----------:|:---------------:|:---------------:|:----------------:|
|   WBSLSW   | 0.9068(0.08744) | 0.3596(0.22447) |  0.3849(0.23038) |
|    ECP30   | 0.9156(0.04346) | 0.9580(0.05754) |  0.9737(0.01215) |
| DPHMM      | 0.9637(0.02257) | 0.8727(0.03066) | 0.8869(0.02651)  |
| KCP        | 0.9501(0.05622) | 0.8436(0.01631) | 0.8836(0.04664)  |
| $D_m$-BOCD | 0.8123(0.11902)  | 0.8411(0.01894) | 0.8413(0.02208) |
| TV-HMM     | 0.9523(0.05174) | 0.9756(0.01339) | 0.9615(0.02004)  |

Reviewer comment (Theory):
How is the transition matrix posterior updated with the ARD prior? Specifically, why line 8. of Algorithm 1 ensures that the updated estimates are in a simplex?

Author response:
The prior is initialized using an upper triangular matrix. In each row, entries are assigned with equal probability summing up to 1. Based on the concept of ARD prior, we investigate that the optimal $\Pi$ after differentiating the ELBO with respect to $\Pi$ is  the transition probability of the previous step $Q^S(t_k(n)=1|t_{k-1}(m)=1)$, which ensures the update estimates are in a simplex

Reviewer comment (Experiments):
How does the proposed method perform compared to using Dirichlet priors on \Pi?

Author response:
Thank you very much for pointing out the symbol issue. We have changed the symbol of the step size of $\tau$ into $\eta$. As mentioned before, each row of the initialized prior is uniformly distributed, with equal probabilities assigned to each element within the row. Our proposed algorithm can update the $\Pi_k$ iteratively based on the data. The specific entry $\pi_{k,m,n}$ of the ARD prior $\Pi_k$ measures the importance of the $k$-th change point transitioning from position $m$ to position $n$, which can be inferred from the data. The central concept of ARD prior is that the hyperparameters (in our cases, $\Pi_k$) are typically optimized by maximizing the marginal likelihood of the data.  Applying Dirichlet priors to $\Pi_k$ can yield similar results to iteratively updating $\Pi_k$. However, this approach might diminish the interpretability of  $\Pi_k$ , as $\pi_{k,m,n}$ becomes a random variable and can no longer be interpreted as a measure of importance or relevance. Moreover, when $\pi_{k,m,n}$ is treated as a random variable, it becomes less straightforward to visualize the sparsity of $\Pi_k$ directly in the plot.

Reviewer comment (Writing):
\tau is used to denote change points (Sec 2) and also the step size in Algorithm 1.

Author response:
Thank you very much for pointing out the symbol issue. We have changed the symbol of the step size of $\tau$ into $\eta$. As mentioned before, each row of the initialized prior is uniformly distributed, with equal probabilities assigned to each element within the row. Our proposed algorithm can update the $\Pi_k$ iteratively based on the data. The specific entry $\pi_{k,m,n}$ of the ARD prior $\Pi_k$ measures the importance of the $k$-th change point transitioning from position $m$ to position $n$, which can be inferred from the data. The central concept of ARD prior is that the hyperparameters (in our cases, $\Pi_k$) are typically optimized by maximizing the marginal likelihood of the data.  Applying Dirichlet priors to $\Pi_k$ can yield similar results to iteratively updating $\Pi_k$. However, this approach might diminish the interpretability of  $\Pi_k$ , as $\pi_{k,m,n}$ becomes a random variable and can no longer be interpreted as a measure of importance or relevance. Moreover, when $\pi_{k,m,n}$ is treated as a random variable, it becomes less straightforward to visualize the sparsity of $\Pi_k$ directly in the plot.

Reviewer comment (Evaluation):
No improvement with respect to competitors.

Author response:
Thank you very much for your suggestion. We also test the semi-parametric TV-HMM on a sequence with piecewise linear segments to show the improvement. In this experiment, the data are generated from the model $\beta_0+\beta_1*t+\epsilon$, $\epsilon$ follows a normal distribution with mean 0 and variance 1. The length of the sequence is 600 with 4 inequal distanced change points. The change point locations are $\tau=[ 80 , 230, 330, 480]$ indicating lengths of each segments are $[ 80,150,100,150,120]$. $\tilde{K}$ is 10. The $\beta_0$ and $\beta_1$ are shown in the following table. 

Model: $y(t)=\beta_0+\beta_1*t+\epsilon$,  $\epsilon \sim N(0,1)$\
Total sample size: $N=600$\
Change point location: $\tau=[ 80 , 230, 330, 480]$\
Initialized number of change point: $\tilde{K}=10$

|           | Segment 1 | Segment 2 |  Segment 3 |  Segment 4 | Segment 5 |
|----------:|:---------:|:---------:|:----------:|:----------:|:----------:|
| $\beta_0$ |     0     | $seg_1(80)$ |$ seg_2(150)$ | $seg_3(100)$ | $seg_4(150)$ |
| $\beta_1$ |    0.1    |    -0.3   |     0.3    |    -0.2    |    -0.2    |

 $seg_i(j)$ stands for the $j$-th element in $i$-th segment.\
\
The summarized average Rand Index and its standard deviation are shown in the following table. It can be observed that Semi-parametric TV-HMM outperforms the other compared methods, indicating that our proposed method can handle the change point detection problem in sequences with linear trends. Moreover, the standard deviation of the Randx Index measurement is 0.02512, demonstrating the stability of our method.
|            |   semi-TVHMM   |      DPHMM     |     ECP3O    |      WBSLSW      |       KCP      |   $D_m$-BOCD   |
|-----------:|:--------------:|:--------------:|----------------|:--------------:|:--------------:|:--------------:|
| Rand Index |      0.9533    |      0.7882    |      0.8580    |      0.2209    |      0.8199    |      0.7637    |
|     SD     |     0.02512    |     0.06822    |     0.00422    |     0.05569    |     0.00197    |     0.09845    |

Reviewer comment (Theory):
Is assumption 2 strong with respect to literature?

Author response:
Thank you very much for your advice. The assumption A1 describes the order of each change point, and the last change point is fixed at $T$ where the first change point $T_1$ is after 0. We also assume the data $y$ in different segments are drawn from a normal distribution with different parameters depending on the regimes. For assumption A2, we assume the number of observations within a specific interval is of polynomial order in the total number of samples, N. Lastly, assumption A3 is about the characteristics of initialized change points. Equally distanced initial change points can be separated into junction and non-junction points. The junction point $t_i$ means there is a true change point within the interval $[t_{i-1},t_i]$. Junction and non-junction points are also demonstrated in Figure 1. 

While the number of observations within an interval is assumed to be polynomially related to the total sample N, this assumption is considered acceptable for developing the algorithms. Let $\alpha = (n-m)/T$, clearly $\alpha$ is less than or equal to 1, which means the rate is slower than N as N approaches infinity. For example, Chakar et al. (2017) prove their change point consistency as N approaches infinity. Thus, our assumption A2 is mild with respect to literature.


Reference

S. Chakar. E. Lebarbier. C. Lévy-Leduc. S. Robin. "A robust approach for estimating change-points in the mean of an AR(1) process." Bernoulli 23 (2) 1408 - 1447, May 2017