Exploiting Representation Curvature for Boundary Detection in Time Series

Authors: yooju  shin
jaehyun  park
hwanjun  song
susik  yoon
byung suk lee
jae-gil  lee
  kaist
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-12 07:05:14

Abstract: *Boundaries* are the timestamps at which a class in a time series changes. Recently, representation-based boundary detection has gained popularity, but its emphasis on consecutive distance difference backfires, especially when the changes are gradual. In this paper, we propose a boundary detection method, **RECURVE**, based on a novel change metric, the ***curvature*** of a representation trajectory, to accommodate both gradual and abrupt changes. Here, a sequence of representations in the representation space is interpreted as a trajectory, and a curvature at each timestamp can be computed. Using the theory of random walk, we formally show that the mean curvature is lower near boundaries than at other points. Extensive experiments using diverse real-world time-series datasets confirm the superiority of RECURVE over state-of-the-art methods.

Main paper: 
 introduction: 
 In a time series composed of sequential data points (simply points) indexed by timestamps, there are boundaries (or change points) signifying transitions between different classes or states, such as a shift from running to walking [1, 2] . Detecting boundaries is a crucial task in preprocessing and diverse applications of time-series data. As preprocessing, they partition a time series into segments of coherent points, accelerating annotation of the time-series for further analysis and giving additional supervision in classification [3, 4, 5, 6, 7] . As primary tasks, they are valuable for identifying changes that require human attention in a variety of domains, including climate, health care, finance, and manufacturing; epilepsy detection, stock price tracking, and action segmentation are examples of possible applications [8, 9, 10, 11] .Representation-based boundary detection methods [12, 13] are prevalent today because they do not require specific assumptions on time-series properties, such as distribution or temporal shape, and can handle high dimensionality due to the capability of a self-supervised model that autonomously learns distinctive features from raw time series without any supervision. In these methods, a self-supervised model [14, 15, 16] is first used to derive a representation of each point, and then a point is identified as a boundary if its representation significantly deviates from those of adjacent points. Let's refer to the points close to a boundary as inter-segment points and the remaining points as intra-segment points. In short, these methods operate by assuming that the distance between consecutive representations is greater between inter-segment points than between intra-segment points.However, this assumption on the distance difference does not always hold, especially when the changes are subtle or gradual. Time-series representation learning methods often pursue preserving the temporal coherence of a time series as their training goal is to make temporally close points similar in their representations and distant points dissimilar [14] . As demonstrated in Figure 1 , the consecutive distances are not clearly distinguishable between intra-and inter-segment points for relatively subtle changes with stair up ↔ stair down because just the direction of motion differs between the two classes, whereas they are for abrupt changes with stand ↔ sit. Thus, the inability to handle subtle changes hinders achieving an overall good performance.In this work, we take a novel perspective on detecting boundaries by leveraging curvatures instead of distances in the representation space. As shown in Figure 2 , the curvature at a point in a curve measures the instantaneous rate of direction change, or more precisely, the amount by which the curve deviates from being a straight line [17] . Suppose that a sequence of point representations from a time series constitutes a representation trajectory. We observe that, regardless of whether the changes are gradual or abrupt, the direction of the representation trajectory tends to change more sharply (showing a higher curvature) at intra-segment points than at inter-segment points. Accordingly, we contend that the curvature of a representation trajectory should be a very promising indicator for class boundary detection.Using Figure 2 , we justify the intuition behind curvature-based boundary detection. Because representation learning tries to learn class-separated features, well-embedded points of a certain class (or a segment) can be drawn from its class-specific ball [18, 19] . That is, the representation trajectory of intra-segment points is confined within a class ball, whereas that of inter-segment points is not. Then, for intra-segment points to reside exclusively within a class ball, their representation trajectory needs to make sharp turns frequently. In contrast, the transition from one class ball to another does not necessarily make sharp turns. This observation is formally proven by the relationship between the mean curvature and the radius of a confining hypersphere, assuming a random walk of a point representation (see Section 3.4) .Overall, for boundary detection, treating a sequence of representations as a trajectory and measuring its curvature is an entirely novel approach, which results in RECURVE (Representation trajEctory CURVaturE). A representation trajectory is derived by a time-series representation learning method, and the curvature at each point is calculated very efficiently; then, the points whose curvature is relatively small are identified as boundaries. RECURVE is simple yet powerful, and can be combined with any time-series representation learning method. We conduct comprehensive evaluations on a variety of time-series datasets, comparing it against state-of-the-art boundary detection methods. The results demonstrate that RECURVE consistently enhances the accuracy, achieving improvements of up to 12.7%. Furthermore, this superiority is shown to exist regardless of the degree of change between different classes. 
boundary detection: 
 Time-series boundary detection methods assess the dissimilarity between two successive intervals and apply a threshold to pinpoint the positions of boundaries. There are multiple methods available for quantifying dissimilarity: (1) conducting statistical tests, (2) quantifying the deviation from discovered patterns, and (3) calculating distances between the representations learned from a selfsupervised model. We summarize each category here, with additional in-depth details available in extensive surveys [1, 2] .Statistical tests often rely on the probability density ratio of two consecutive intervals as a key statistic. CUSUM is a traditional parametric algorithm that adds up the log likelihood ratio when a probability density function is given [20, 21] . RuLSIF is a non-parametric algorithm that directly estimates the probability density ratio using Pearson divergence without a probability density function [22, 23, 24] . A kernel-based statistical test maps each interval to a kernel space and then computes the kernel Fisher discriminant ratio as a statistic [25] . KL-CPD uses a deep neural network as a generator for kernel parameters, which solves high sensitivity in selecting parameters [26] .The proactive discovery of frequent temporal patterns is necessary for temporal pattern-based boundary detection. FLOSS stores the locations of similar subsequences in a time series using Matrix Profile and measures the likelihood of a regime change [27] . Motif-based boundary detection relies on the identification of short temporal patterns (motifs) determined through the minimum description length criterion; these motifs are then compared for similarity with other subsequences within a time series [11, 28] . ESPRESSO, on the other hand, is a hybrid of pattern-and statistic-based approaches, detecting a wide range of boundaries across different scenarios and data types [29] .Representation-based boundary detection methods are distinguished by the manner in which a self-supervised model is trained. TIRE exploits an autoencoder to retain time-invariant features in consecutive timestamps to make representations of boundaries salient [12] ; after training, the output representations undergo a process of smoothing, wherein a moving average is applied prior to the dissimilarity computation. TS-CP 2 leverages contrastive learning techniques to promote close proximity between representations of two consecutive timestamps and distant proximity between representations at randomly selected timestamps [13] ; it examines the difference between each consecutive distance and the moving average.Time-series representation learning builds a model to create versatile representations capable of performing diverse downstream tasks such as classification, forecasting, and anomaly detection [30, 31] . Reconstruction-based learning methods train autoencoder-based deep neural networks using a reconstruction loss. TimeNet is an early example that uses a sequence-to-sequence autoencoder and uses the hidden embedding extracted from the encoder as a representation [32] . DTCR extends traditional reconstruction-based learning by incorporating a k-means loss alongside the reconstruction loss [33] . Input masking is also commonly used for reconstructing data with specific timestamps intentionally masked or hidden [34, 35, 36] .In contrastive learning, the Info-NCE (Noise Contrastive Estimation) loss plays a pivotal role by bringing a positive pair closer together and pushing a negative pair apart in the representation space. An early approach considers a sampled window and a subsequence from the window as a positive pair [37] . In recent methods such as TNC (Temporal Neighborhood Coding), the temporal distance serves as a criterion for identifying a positive pair, keeping two neighboring timestamp representations close [13, 14, 38] . Following the principles of SimCLR [39] , a positive pair can be created by pairing a sampled window with its augmentation which involves data perturbation or context changes [15, 40] . Besides, the Fourier transform of a time series serves as an augmentation technique for generating positive pairs or providing a new representation space [16, 41, 42] . 
preliminaries and problem setting: 
 Dataset and Model: Let X = (x t ) T t=1 be a time series, where T is the total number of points, andx t ∈ R d is a d-dimensional point at timestamp t. Let C = {t k | k ∈ 1, K } bea set of the timestamps for the ground-truth boundaries. Considering class labels annotated at each timestamp, C is composed of the timestamps where there is a change in the label from the previous one (e.g., stand → walk). A window X tm = (x t ) tm+I-1 t=tm-I is a sequence of consecutive 2I points centered at timestamp t m . A representation model f θ , which is a deep neural network parameterized by θ, converts each window X tm to its representation z tm ∈ R d ′ , i.e., z tm = f θ (X tm ).Representation Learning: RECURVE is not bound to a specific representation learning method, and we summarize the training process using one of the popular methods, the temporal predictive coding (TPC) proposed in TS-CP 2 [13] . Here, two non-overlapping consecutive windows are used as a positive pair, and two randomly-sampled windows are used as a negative pair. Thus, TS-CP 2 randomly samples b windows as well as their succeeding windows and constructs a batch B = {X t1 , X t2 , . . . , X t b , X t1+2I , X t2+2I , . . . , X t b +2I }; the method subsequently minimizes the InfoNCE loss ℓ NCE [43] ,EQUATIONwhere sim(•, •) is the cosine similarity function, exp(•) is the exponential function, z tj = f θ (X tj ), and τ is a scaling parameter. The model parameter θ is updated iteratively by gradient descent, i.e., θ ← θ -η∇ θ ℓ(B, θ), where η is a learning rate.Change Metric and Detection: Using the representations of all windows centered at each point in X , i.e., {z t | t ∈ 1, T }, a change metric ŷt is derived for each point x t ∈ X , which represents the extent that x t is a boundary. For example, the change metric in TS-CP 2 employs the distance (i.e., cosine similarity) between the embeddings of adjacent points,EQUATIONwhere MA calculates a simple central moving average and NORM is min-max normalization over all timestamps to rescale a value between 0 and 1. Then, similar to binary classification, the points whose change metric exceeds a certain threshold φ are identified as boundaries, Ĉ = {t | ŷt ≥ φ where t ∈ 1, T }.(3) Goal: Obviously, an effective change metric is crucial to the success of boundary detection. Therefore, we propose a novel change metric, ŷcurv t , using the curvatures in the representation space instead of the consecutive distances in the representation space.A trajectory usually refers to the path or track that an object (e.g., human and vehicle) in motion follows through space and time [44] . Thus, we get to Definition 3.1 if we think of an object as a point floating in the representation space. The curvature at a specific point on a curve is the rate at which the direction of the curve changes instantaneously at the point [17] . It is a well-defined concept in geometry and quantifies how sharply or gradually the curve bends or deviates from a straight line. We employ the definition designed for a trajectory [45] . For three timestamps in order, t -, t, and t + , where t -< t < t + , consider their representations z t -, z t , and z t + . Two difference vectors, z t -z t -and z t + -z t , are naturally derived, and the turning angle θ t between them in Figure 3 is calculated byEQUATIONEach value of θ t ranges between 0 and π, where t ∈ 2, |T |-1 . Then, the curvature is the rate of the direction changes between the two difference vectors, i.e. how much a difference vector rotates per unit length, as defined in Definition 3.2.. The curvature at timestamp t in a representation trajectory T is the turning angle θ t divided by the sum of the difference vector lengths,EQUATION)According to our observations and intuitions described in Section 1, the curvature of an intra-segment point is higher than that of an inter-segment point. Thus, the curvature defined in Definition 3.2 can be used as a change metric. The computational complexity remains O(d ′ T ), consistent with that of TS-CP 2 where cosine similarity is computed instead of curvature. For stability, the timestamps t -and t + in Eq. ( 5) are determined to be w ≥ 1 timestamps before and after timestamp t, i.e., t -= t -w and t + = t + w. We set w to 5% of the mean segment length, which is observed to work well in most situations. Please refer to Section 4.6 about the sensitivity analysis on the value of w. Definition 3.3 concludes our novel curvature-based change metric.). The curvature-based change metric at timestamp t becomes ŷcurvEQUATIONwhere κ t is obtained from Eq. ( 5) and MA and NORM are the same as Eq. (2). In Definition 3.3, we normalize the curvature to a scale of 0 to 1. Subtracting this normalized curvature from 1 ensures that the change metric increases as the curvature decreases while maintaining the scale. Finally, a moving average is employed to smooth the curvature and mitigate fluctuations.Once the change metric ŷcurv t in Eq. ( 6) is prepared, it is possible to detect boundaries by finding the points where ŷcurv t ≥ φ, as formulated by Eq. ( 3). Therefore, it is necessary to develop a heuristic for determining the threshold φ, and additional information can be utilized for this purpose. Such additional information includes the mean segment length and the validation dataset. If the mean segment length, i.e., the average of the lengths of segments distinguished by boundaries, is known, the estimated number of boundaries can be calculated by dividing the total number of timestamps by the mean segment length. The threshold φ is then determined to obtain the estimated number of boundaries. Alternatively, if we have a validation dataset, we select the threshold φ that yields the best performance based on an evaluation measure. Empirical evaluation in Section 4 employs the mean segment length in thresholding. Our theoretical analysis is conducted by showing the following properties: (1) the intrasegment points in the representation space are confined within a smaller hypersphere than the inter-segment points, as shown in Figure 4;  (2) the mean curvature of a representation trajectory increases as the radius of the confining hypersphere decreases, which leads to the rationale behind Definition 3.3.Definition 3.4 (CONFINEMENT). Consider a subsequence of a specific class C i , X Ci = (x t ) t endt=tstart , as well as its representation trajectory, T Ci = (z t ) t end t=tstart , in Figure 4 . Then, T Ci is confined within a hypersphereS Ci ⊂ R d ′ centered at O Ci ∈ R d ′ of radius R intra if and only if ||z t -O Ci || < R intraholds for all t ∈ t start , t end . Definition 3.4 comes from a widely known fact that representation (contrastive) learning produces class-separated representations [18, 19] . The augmented positive examples form a connected graph based on augmentation overlap; thus, the alignment of positive examples by contrastive learning will cluster the examples of the same class together and lead to class-separated representations [18] . Proposition 3.5 (CONFINEMENT RADIUS). Consider a transition from a class C i to another class C j in Figure 4 . Let S Ci , S Cj ⊂ R d ′ be the confining hyperspheres for C i and C j , respectively, of radius R intra . Then, consider a larger hypersphere of radius R inter that encloses the inter-segment points (in red) as well as S Ci and S Cj . Thus, R intra < R inter holds by definition.Based on temporal coherence [7, 13, 46] inherent in time series, we make an assumption on the representation trajectory before proceeding to the second step.|T | t=1 is a Markov chain, where z t is sampled over the surface of the unit hypersphere centered at z t-1 and also contained in a confining hypersphere of radiusR. That is, ||z t -z t-1 || = 1 (t ∈ 2, |T | ) and ||z t || < R (t ∈ 1, |T | ) such that R > 1.Under Assumption 3.6, the curvature in Eq. ( 5) becomes the turning angle in Eq. ( 4) because the denominator is reduced to a constant when w = 1. Then, when a given representation trajectory T is confined by a hypersphere of radius R, its mean curvature is defined byEQUATIONwhere E zt|R [θ t ] is the expectation of the curvature at timestamp t with respect to the distribution of the representations in the confining hypersphere of radius R.Lemma 3.7 (MEAN CURVATURE). Consider a representation trajectory T confined in a hypersphere of radius R under Assumption 3.6. Then, the mean curvatureK T (R) is a decreasing function of the radius R, i.e., d dR K T (R) < 0.The proof of Lemma 3.7 is provided by Diao et al. [47] . The mean curvature is rigorously formulated as a complicated integral. By a simulation of random walk with one million steps, the decrease in the curvature is represented by the linear function 3.53 -1.21R and the function π/2 + 0.65/R 1.5 for two different regimes of R.Notation. The representation trajectories confined within the hyperspheres of radii R intra and R inter in Figure 4 are called intra-segment and inter-segment trajectories as well as denoted by T intra and T inter , respectively.Putting Proposition 3.5 and Lemma 3.7 together, the observation on the difference in the curvature is finally formalized by Theorem 3.8.The mean curvature of an intra-segment trajectory is greater than that of an inter-segment trajectory, i.e., K Tintra (R intra ) > K Tinter (R inter ).Proof. Because R intra < R inter by Proposition 3.5, K Tintra (R intra ) > K Tinter (R inter ) obviously holds by the decreasing nature of K T (R) of Lemma 3.7.Theorem 3.8 can be intuitively explained if we consider the special case in which the next representation of z intra or z inter lies on the surface of a hypersphere, as visualized in Figure 4 . Since a smaller radius necessitates a sharper turn, θ intra > θ inter holds true. In this particular instance, where z intra or z inter is an orthogonal projection onto the surface, the turning angle can be expressed as πarccos 1 2R , a decreasing function of R. Please refer to Appendix A for more details.The findings in the theoretical analysis also align well with the visualizations of the representations from a real dataset. 1 , where the window size is approximately twice the sampling rate. The learning rate is set to 0.005 for all datasets. The hyperparameter w, indicating the length of a representation vector, is set to 5% of the mean segment length. After obtaining a change metric for each timestamp, in Eq. ( 6), the same normalization is applied to all windows from an individual dataset. The moving average in Eq. ( 6) is computed using the ten timestamps preceding and following each timestamp. RECURVE is implemented using PyTorch 1.13.0, and its source code is available at https://github.com/kaist-dmlab/RECURVE.Compared Methods: RuLSIF [24] , KL-CPD [26] , and TS-CP2 [13] are chosen as the representative method from each of the three categories in Section 2.1. The window size in Table 1 is applied to all compared methods for fair comparison. A multilayer perceptron is used for the regressor of RulSIF. The hyperparameters of RuLSIF and KL-CPD are favorably determined by a grid search, as detailed in Appendix B. The public implementations of RuLSIF 2 and KL-CPD3 are used for our experiments. TS-CP 2 is the closest to our work, and its main mechanism is briefly described in Section 3.1. Because representation learning itself is shared between TS-CP 2 and RECURVE when TPC is used, the same hyperparameter setting is applied to both methods whenever possible. TS-CP 2 is re-implemented using PyTorch 1.13.0 for direct comparison with RECURVE.Evaluation Measures: First, the Area Under the ROC Curve (AUC) is measured by considering boundary detection as binary classification with a binary label vector y ∈ {0, 1} T converted from C. Following Deldari et al. [13] , an error margin is introduced to accommodate some noise from annotation and detection. A detected boundary is considered to be correct if it lies within p timestamps from one of the ground-truth boundaries. For this purpose, y is relaxed toEQUATIONThen, for t ∈ 1, T , whether (ŷ t in Eqs. ( 2) or (6) ≥ φ) is compared against y t in Eq. ( 8). We use multiple error margins, p ∈ {5, 10, 20}, since a margin could be different for diverse applications [1] . Second, the mean LOCation distance (LOC) is measured, which is the average distance from a detected boundary to its closest ground-truth boundary [27, 51] . The LOC measure is useful for checking the preciseness of the boundaries detected.Regarding the threshold φ, the AUC measure does not require a specific value because it evaluates the true positive and false positive rates over a given range. For the LOC measure, two values are used for each experiment: one is determined to achieve the best F1 score, and the other is determined by the heuristic based on the mean segment length in Section 3.3, where the estimated number of boundaries is multiplied by p = 10, taking the error margin into account.For each evaluation measure, we conduct every experiment five times with different seeds and report the average as well as the standard deviation. We use Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz and NVIDIA RTX 3090 for every experiment. This periodicity would produce a closed shape for intra-segment trajectories and increase their curvatures in the representation space, enhancing the performance of RECURVE. In particular, when p = 20, RECURVE outperforms the second-best method by up to 12.7% in terms of the AUC measure for the mHealth dataset.  
detailed investigation on change metric quality: 
 We display the average values of the change metrics separately for each pair of classes using the HAPT dataset, which was chosen for ease of visualization due to its small number of classes. 2 and 3 . In contrast, in Figure 6b , ŷdist t only generates high values when the inter-class embedding distance is sufficiently large (i.e., abrupt change), whereas it generates moderate values when the inter-class embedding distance is small (i.e., gradual change). That is, Figures 6a and 6b show a very high correlation. In summary, ŷcurv t is insensitive to the degree of changes whereas ŷdist Table 4 shows the corresponding AUC scores for gradual and abrupt changes in relation to various distance-based metrics. If the inter-class embedding distance between two classes is shorter than a certain threshold, the change between the two classes is categorized as gradual. Here, the threshold is established such that gradual changes represent 20% of all changes. Abrupt changes are excluded when measuring the improvement for gradual changes, and vice versa. The distance-based metrics, denoted by DISTANCE, are further categorized depending on whether the Euclidean distance or the cosine similarity is used. Note that RECURVE adopts the curvature-based metric.Overall, RECURVE demonstrates greater improvements for gradual changes (AUC-Gradual) than for abrupt changes (AUC-Abrupt). For example, in the HAPT dataset with p = 5 and the TPC representation, the increase in the AUC measure from TS-CP 2 to RECURVE for gradual changes is 32%, which is significantly higher than the 21% increase for abrupt changes. This result confirms that the curvature-based method is particularly effective for detecting gradual changes because it captures subtle variations in the data that are not readily discernible through distance-based methods. In conclusion, RECURVE is versatile to support both gradual and abrupt changes. 5 0.807±0.014 0.602±0.023 0.546±0.015 0.671±0.010 0.838±0.009 0.746±0.021 0.631±0.011 0.703±0.039 10 0.811±0.014 0.606±0.023 0.548±0.014 0.674±0.010 0.845±0.010 0.747±0.021 0.636±0.011 0.706±0.035 20 0.814±0.015 0.613±0.022 0.550±0.011 0.677±0.009 0.847±0.010 0.747±0.020 0.637±0.010 0.714±0.037 TNC 5 0.779±0.013 0.774±0.013 0.789±0.017 0.594±0.022 0.794±0.012 0.945±0.006 0.892±0.023 0.681±0.010 10 0.787±0.012 0.783±0.012 0.803±0.015 0.594±0.032 0.808±0.011 0.946±0.004 0.901±0.022 0.685±0.010 20 0.814±0.013 0.799±0.014 0.846±0.009 0.601±0.032 0.846±0.011 0.950±0.004 0.928±0.017 0.687±0.010 RECURVE TPC 5 0.888±0.004 0.886±0.009 0.939±0.003 0.712±0.006 0.923±0.003 0.945±0.002 0.985±0.004 0.729±0.055 10 0.893±0.005 0.891±0.009 0.940±0.003 0.715±0.006 0.927±0.002 0.948±0.002 0.988±0.004 0.731±0.054 20 0.894±0.005 0.896±0.009 0.941±0.003 0.722±0.006 0.927±0.002 0.955±0.002 0.988±0.004 0.734±0.051 TNC 5 0.867±0.007 0.773±0.038 0.975±0.003 0.551±0.034 0.901±0.006 0.931±0.002 0.988±0.006 0.600±0.016 10 0.875±0.007 0.779±0.038 0.977±0.003 0.553±0.034 0.910±0.006 0.933±0.002 0.989±0.006 0.602±0.016 20 0.890±0.008 0.791±0.038 0.977±0.004 0.558±0.033 0.927±0.006 0.939±0.002 0.990±0.006 0.606±0.015Figure 8 visualizes the fluctuations of various change metrics obtained from the HAPT dataset using the TPC representation. TS-CP 2 fluctuates rapidly during the changes and seems to have many false negatives at the rightmost boundary area. However, RECURVE indicates the inter-segment points much more clearly than TS-CP 2 , without excessive false positives and false negatives. The reliable detection is achievable by taking into account both the turning angle (the numerator in Eq. ( 5)) and the distance (the denominator in Eq. ( 5)) in the representation space. Table 5 shows the performance of RECURVE while varying the hyperparameter w (see Definition 3.3) when the error margin p for the AUC measure is fixed at 10. The value of w ranges from 0.25× to 4.00× of the default value, which is set to 5% of the mean segment length (indicated by 1.00×).If the value of w were too large, the denominator of Eq. ( 5) would be too large for any point in a time series, and the curvature would be unable to distinguish between intra-and inter-segment points.If the value of w were too small, some noise in point representations would distort the curvature. Under this trade-off, the default value performs the best in terms of the AUC measure when it is averaged over the four datasets and the two representation learning methods. On a dataset with lengthy segments, such as mHealth, the sensitivity tends to decrease, and there is small variation when varying the value of w. The sensitivity analysis on the representation dimensionality d ′ is available in Appendix C.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)? Answer: [Yes] Justification: Section 4.1 contains dataset summary and implementation details for reproducing main experimental results. Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 
conclusion: 
 In this paper, we present RECURVE, a novel boundary detection method that uses the curvature of a representation trajectory to replace the consecutive distance for a change metric. Theoretically, the mean curvature of an intra-segment trajectory is greater than that of an inter-segment trajectory due to the confining nature of the representations of the points within a single class. Unlike the consecutive distance, this property of the curvature is insensitive to the degree of the changes between two classes (segments). Our comprehensive experiments confirm that RECURVE achieves up to 12.7% higher detection accuracy than state-of-the-art methods. Overall, we believe that our work pioneers a new direction for boundary detection in time series.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: Section 4.1 contains a link for source code and datasets. Guidelines:• The answer NA means that paper does not include experiments requiring code. Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes] Justification: This paper does not have any potential harms or such consequences.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [NA] Justification:Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [NA] Justification: Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. 
 
 Appendix: To explain Theorem 3.8 in a more intuitive way, we assume a special case in the 2-dimensional space as shown in Figure 4 . Let we have a random walk composed of three points z t -, z t , and z t + in a 2-dimensional representation space. In the figure, there are three black dots in a row for the intra-segment trajectory (gray line) and the inter-segment trajectory (red line) respectively to visualize these three points in each trajectory. Here, z t is an arbitrary point on the border of a confining circle S Ci or the enclosing circle. z t -is the point on the line between the origin and z t . Finally, we consider z t + as a random point distributed uniformly on the unit circle centered at z t while the locations of z t and z t-1 are fixed. In this case, we compare the average curvature at the single timestamp t of the intra-segment trajectory and that of the inter-segment trajectory. Lemma A.1 (AVERAGE CURVATURE). When the two points z t -and z t are fixed, the curvature κ at z t averaged over the location ofEQUATIONProof. When the confinement radius is R, the range of the turning angle is [πarccos 1 2R , π] using the cosine rule on the triangle composed of the points of intersection between S Ci and the unit circle centered at z t .Lemma A.2 (CHARACTERISTIC OF AVERAGE CURVATURE). When R > 1, κ T (R) decreases as the radius R increases, i.e., κ T (R) is a decreasing function of R. Proof. The derivative of κ is d dR κ T (R) = - 1 2R √ 4R 2 -1 and the derivative is always negative when R > 1. Therefore, κ T (R) is a decreasing function of R.Similar to Theorem 3.8, Proposition 3.5 and Lemma A.2 lead to Theorem A.3. Theorem A.3 (AVERAGE CURVATURE DIFFERENCE). The average curvature of an intra-segment trajectory κ Tintra (R intra ) is greater than that of an inter-segment trajectory κ Tinter (R inter ).Proof. The radius of the confining circle of an inter-segment trajectory is always bigger than that of an intra-segment trajectory because an inter-segment trajectory traverses two distinctive class balls. As κ T (R) is a decreasing function, κ Tintra (R intra ) > κ Tinter (R inter ).Theorem A.3 concludes that the average curvature at z t is greater if z t is at the border of a smaller confining circle. This result can be extended to Theorem 3.8 intuitively. As R increases, the probability of a trajectory hitting the border decreases as a random walk can traverse in a larger space. Therefore, the mean curvature decreases if a trajectory resides in a smaller confining circle.For RuLSIF, we conduct a grid search for the learning rate (LR) = {0.05, 0.1, 0.2}, the weight of L2 normalization λ L2 = {0.01, 0.05, 0.1}, and the parameter of the RuLSIF loss α = {0.01, 0.05, 0.1}. When applying RuLSIF to the four datasets, we use a multilayer perceptron with a single hidden layer with 100 units and train it with a batch size of 32 for 50 epochs. For KL-CPD, we conduct a grid search to determine the optimal hidden dimensionality h = {10, 50, 100} of the RNN encoder/decoder, as well as the values for the hyperparameters λ AE = {0.1, 0.01, 0.001} and λ Real = {0.1, 0.01, 0.001} which govern the influence of the reconstruction loss and the MMD2 loss on real datasets. For training the generator of KL-CPD, the batch size is set to 64, the number of epochs is set to 3, and the learning rate is set to 0.001. Table 6 provides a summary of the determined hyperparameter values.Table 7 shows the performance of RECURVE while varying the representation dimensionality d ′ when the error margin p for the AUC measure is fixed at 10. The value of d ′ ranges from 0.25× to 4.00× of the default value, which is 8 for WISDM and HAPT or 32 for mHealth and 50salads (indicated by 1.00×). A trade-off point in the representation dimensionality exists for nearly all datasets. A representation space with an excessively high dimensionality is susceptible to the curse of dimensionality. If the value of d ′ were too large, the turning angle and distance in Eq. ( 5) would be indistinguishable across all timestamps in a time series, as any two points in a high-dimensional space would become nearly orthogonal and their distance would always be similar. If the value of w were too small, low-quality features would be extracted from the original time series by representation learning; thus, the performance degrades with an insufficient dimensionality as shown in the result of 50salads whose data dimensionality is 2048. Overall, the default setting is suitable for achieving competitive performance for all datasets. One notable limitation of RECURVE is the potential occurrence of false positives, particularly when dealing with short segment lengths that fall below the predefined threshold, denoted as w. For instance, consider a scenario where there is a rapid transition from one activity class to another, such as transitioning from walking for an extended period to a brief sprint, followed by resuming walking. In such cases, if the segment length is shorter than the specified w duration, every timestamp within the brief sprint segment might erroneously be identified as a boundary, leading to a number of false positives. To mitigate this issue, we have implemented a strategy where w is determined as 5% of the mean segment length. However, it is worth noting that this approach may encounter challenges in rare cases where the segment lengths exhibit significant variance. We will further investigate how segment length variance impacts the efficacy of RECURVE as future work.Justification: To the best of our knowledge, Section 3.4 contains all of details for the theoretical analysis. Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced.https://github.com/HSE-LAMBDA/roerichhttps://github.com/HolyBayes/klcpd 


Decision: This paper introduces a novel method, RECURVE, which utilizes the curvature of representation trajectories to detect boundaries in time series data. After a comprehensive evaluation, I recommend accepting this paper for the following reasons. The primary strength of this paper lies in its innovative approach to boundary detection. By analyzing the curvature of representation trajectories rather than relying solely on distance-based metrics, the method effectively handles both gradual and abrupt changes in time series data. This approach addresses a critical gap in existing methods, which often struggle with gradual transitions. The authors provide strong theoretical foundations for their approach, supported by rigorous empirical validation across diverse real-world datasets, where RECURVE consistently outperforms state-of-the-art methods. The paper is also well-structured and clearly written, making it accessible to a broad audience. The authors have responded effectively to reviewer concerns, particularly in demonstrating the necessity of the curvature metric and expanding their experimental comparisons to include additional baselines. They have also provided thorough explanations of the limitations and potential areas for future work, such as exploring class-separated representations further. While some concerns remain about the dependency on the quality of learned representations and the need for clearer guidance on parameter sensitivity, the overall contribution of the paper is significant. It introduces a new perspective on boundary detection in time series, backed by strong empirical evidence and theoretical insights. The method's robustness across different types of changes and datasets suggests that it has the potential to impact various applications significantly.

Reviews: review: paper_summary: paper_summary: The paper introduces a boundary detection method called RECURVE for time series data, which leverages the curvature of representation trajectories as a novel change metric to accommodate both gradual and abrupt changes. Experiments on diverse real-world datasets demonstrate that RECURVE outperforms state-of-the-art methods, achieving up to a 12.7% improvement in detection accuracy.
strength_weakness: strength_weakness: 1. While the concept introduced in this paper is highly innovative—it detects boundary points from the perspective of representation trajectories— the intrinsic advantages of using a curvature metric have not been fully articulated: 

- 1.1 Although the paper proposes a method based on the trajectory perspective, it appears to rely primarily on data from three points in the trajectory (t+1, t, t-1), which leads to a final metric design that is similar to distance-based methods. 

- 1.2 Looking at Equation 3.2, there is a certain correlation between the curvature-based method and the distance-based method: k_t is negatively correlated with distance. This raises the question of whether the curvature-based method might actually subsume the distance-based method. Is the introduction of the turning angle theta truly necessary? The authors should delve into what the RECURVE method solves that distance-based methods cannot. 

- 1.3 Why is the curvature of intra-class points so large? Is this due to an excessively large theta or a too-short distance? Similarly, why is the curvature of inter-class points so small? Is this due to a too-small theta or a too-long distance?

2. Assumption 3.6 raises some concerns. In this assumption, the authors posit that ||z_t − z_{t−1}|| equals 1. However, this assumption appears to be difficult to uphold in boundary point detection tasks; otherwise, distance-based methods should be universally ineffective. If this assumption does not hold, what are the implications for the perspectives expressed in this paper?

3. Minor Question: The definition of Mean Total Curvature is somewhat perplexing; could you clarify why it is termed "Mean Total" rather than simply "Mean"?

4. Other Inquiry: I have some queries regarding the Case Study. In Figure 5(c), I have manually marked five points: a, b, c, d, and e (you can view this through [this link](https://anonymous.4open.science/r/temp-AC2C/5c.png)). The curvature at point b, denoted as k_b=\theta_b/(ab+bc), and at point d, k_d=\theta_d/(cd+de). Clearly, \theta_b is an obtuse angle and \theta_d is an acute angle, implying \theta_b > \theta_d. Additionally, visually assessing the distances on the image suggests that (ab+bc)<(cd+de). Consequently, it would be expected that k_b>k_d. Considering that a smaller k indicates an inter-segment point, why does Figure 5(c) depicts b as an inter-segment point and d as an intra-segment point. Am I misunderstanding something here?
questions: questions: See weakness. I'd be happy to raise my score if my questions were answered well.
limitations: limitations: Yes.
paper_summary: paper_summary: This paper proposes a novel boundary detection method based on the curvature of a representation trajectory. The feasibility of the proposed algorithm is analyzed intuitively and theoretically, and the proposed method is experimentally shown to have good performance.
strength_weakness: strength_weakness: It may be better to first classify the boundaries into several categories, and then discuss the related work and experimental comparison for these categories respectively.
questions: questions: 1. How many types of boundaries are there?
2. Which boundary problems can the proposed method solve and which boundary problems cannot it solve?
limitations: limitations: Yes
paper_summary: paper_summary: This paper proposed to use curvature as the metric to detect the boundaries. Empirical and experimental results also show that the confining box is different for inter- and intra- variable data points.
strength_weakness: strength_weakness: - The paper lacks a strong foundational motivation for introducing the curvature-based boundary detection method. The rationale behind why curvature would be a better metric compared to existing methods is not convincingly argued. From Figure 5, it seems that the distance-based methods can also split (a) and (c) well while for (b), the curvature difference is also not very big.
- Many related boundary detection methods are mentioned in the related work but not directly compared in the experiment. It would be beneficial to include a more comprehensive comparison to show the effectiveness of the proposed approach.
- RECURVE is highly dependent on the quality of the learned representation which makes it also holds the same limitation as a distance-based model when the representation of a time series data is not properly learned.
questions: questions: - How is the sensitivity of the parameter analysed? Are the results presented with the test set or is it with the evaluation set? 
- In Figure 5 (b), the curvature of some continuous intra-variable data points seems to have a similar curvature (approximately a straight line) compared to the curvature between the variables from a different class. Can you clarify this observation and how it may affect the proposed method?
- What is the computational cost of the proposed method compared to baselines?
limitations: limitations: The limitation is well stated in Section D.
paper_summary: paper_summary: The authors propose a novel boundary detection method for time series based on measuring the curvature of the local trajectory of a learned per-timepoint embedding/representation. Some theoretical justifications are provided for the proposed method. Empirically, it is shown to work better than a number of existing methods on a few time series datasets.
strength_weakness: strength_weakness: The whole idea builds on the fact that there exists a learned representation that more or less satisfies the locality assumption (e.g. intra-class points live within a hypersphere). Although some limited empirical evidence is presented, it is unclear in general whether this is a common phenomenon or something else needs to be done to encourage representations that more faithfully respect the assumption.
questions: questions: I wonder whether one can go a step further and establish some notion of optimality under the right settings. Suppose that the representation for each class is drawn from a gaussian with class-dependent mean/covariance, then there exist optimal statistical tests for the inter-class change points. How would the proposed method perform compared to the optimal boundary detector?
limitations: limitations: N/A
paper_summary: paper_summary: This paper proposes a novel approach called RECURVE (Representation trajEctory CURVaturE) for boundary/change point detection based on time series representations. RECURVE measures changes in representations of time series windows over time, based on curvatures instead of distances in the representation space like existing work. The intuition is that the direction of the representation trajectory tends to change more sharply at points within a segment than at points between segments. This is because representation learning aims to learn class-separated features, and intra-segment points are confined within a class-specific ball, while inter-segment points are not. RECURVE works by first deriving a representation trajectory using a time-series representation learning method, then calculating the curvature at each point to identifying boundaries.
strength_weakness: strength_weakness: $\textbf{Class separated representations: }$ The theoretical analysis relies on Definition 3.4 which states that contrastive-based representations learning produces class-separated representations. First, there is no guarantee of class separation for contrastive-based methods, and in many complex datasets this doesn't in fact happen. If the proposed method is only justified under class-separated settings, this should be clearly stated and the authors should propose a way to assess if the representations generated by a contrastive framework qualify for this. Second, In the case of perfect class separation, a clustering method can be used to identify the underlying states for all windows and therefore find the time of transition between the states. Can the authors justify their reasoning regarding this issue?

$\textbf{Normalizing the change metric: }$ A limitation of many change point detection (CPD) methods is that they normalize the change metric in order to obtain a normalized score. This means the largest change in a single time series sample will always be assigned a CPD score of 1, even if the time series sample has no true change in state. This also implies that the value of the CPD scores are not comparable across samples because they are normalized per sample. This limitation applies to the proposed method as well and the authors need to elaborate on this issue more. Aside from that, this problem will also have implication on the choice of threshold used to identify the change points. 

$\textbf{Baselines: }$ An important baseline that is missing from the evaluations is a distance-based change point detection model that uses the same representation learning framework as RECURVE. The main claim of the paper is that the curvature is a better measure of change, so this needs to be evaluated in the evaluation section. I would suggest having baselines similar to RECURVE+TNC and RECURVE+TPC, but only using a similarity metric like cosine similarity or even euclidian distance (DISTANCE+TNC and DISTANCE+TPC). Such baseline can also show the advantage of the proposed method in samples with gradual transition.
questions: questions: $\textbf{Change metric quality: }$ In section 4.3, the authors show that the change metric of RECURVE is on average always high for transition points while distance-based metrics have varying score values depending on the state change. My question is why having varying degrees is a bad thing? Because in many cases, this can also show us the similarity or difference between the two transition states (Walking to sitting vs walking to lying down). 

$\textbf{Scores over time: }$ Please include a figure that shows the estimated CPD scores for a sample over time. This will be very helpful to understand the behaviour of the method, in particular around the change point boundary.

$\textbf{Error margin: }$ The choice of error margin seems to be an important one, especially to show how the proposed method behaves in slow-transition states. Can you provide performance results across multiple error margin values in the appendix and discuss your observation.
limitations: limitations: Overall, the limitations are well-covered in the paper. Some points that I mentioned in earlier sections can be added to this discussion.

Comments: title: Thank you for your feedback
Thank you for your feedback
Thank you for your feedback
Thank you for your feedback
Rebuttal response
Thank you for your feedback
comment: We greatly appreciate your valuable feedback and strong support.
Thanks for the response, I'll keep my score.
We are happy to hear that our responses are satisfactory to you. Thank you again for your valuable comments and suggestions. We will carefully incorporate your comments into the final version.
We greatly appreciate your valuable feedback and encouragement. Your comments will be definitely incorporated into our final version.
Thanks for your responses, I will keep my positive score.
The authors have provided clear and informative responses to my concerns.

I maintain my positive score on this paper.
We are pleased that you find our responses to be satisfactory. Thank you again for your valuable and insightful feedback. We will clearly discuss the issues about class-separated representations and surely add the additional results to the final version.
This rebuttal has been very helpful to clarify my main concerns. I believe the additional results and figures in the appended pdf will be important additions to the paper. 
My first concern about class-separation still exists. I think the authors should definitely discuss this in the paper because it will be an important thing to consider if someone wants to pick a representation learning framework to combine with the proposed method. 
But overall, I'm happy to increase my score based on this rebuttal.
We are pleased that you find our responses to be satisfactory.  Thank you again for your valuable and insightful feedback.  We will incorporate your comments into the final version.
Thanks for the insightful reply. The authors have solved most of my concerns, and I have raised my score.