## 4 Entropy-Isomap  
As we established earlier, standard Isomap does not work well for dynamic process data since neighboring data points are typically from the same trajectory. However, the global structure of the process manifold is determined by the relations between different trajectories. When \(k\)-NN neighborhoods are computed, this can result in poor mixing of the trajectories. The mixing can also vary depending on the temporal location of the process. For example, if the trajectories are generated using similar initial conditions, the trajectories will exhibit strong mixing. However, the trajectories typically diverge subsequently. A neighborhood size \(k\) that produces good results in early stages might produce poor results later on in the process. A value of \(k\) that is large enough to work for all times might include so many data points that the geodesic and Euclidean distances become essentially the same, which results in a PCA-like behavior, defeating the purpose of using Isomap. To address this situation, we propose to directly measure the amount of mixing and use it to change the neighborhood size for different data points adaptively. This mitigates the shortcomings of the two methods described in the previous section, which either discard data (subsampling) or lose local information (skipping). Figure 5 shows that neighborhood entropy increases when the next nearest neighbors are added. We propose using a threshold on the neighborhood entropy, as defined above, to adaptively determine an appropriate neighborhood size, \(k\). This modification allows the flexibility of larger neighborhoods in regions where it is necessary or desired to force mixing between trajectories. An additional parameter, \(M\), determines the largest possible neighborhood size. This user-defined parameter is used to ensure that the neighborhoods are not so large as to reduce Isomap to PCA. For datasets which contain trajectories in poorly sampled regions of the state space, \(M\) controls the size of the local neighborhood, without skewing the rest of the analysis, which would otherwise result in an unreasonably large value of \(k\). Algorithm 2 outlines the step of the proposed Entropy-Isomap algorithm. While the proposed algorithm has some similarities to the standard Isomap algorithm (See Algorithm 1), there are some key differences. First, the algorithm takes two additional arguments, the target entropy level, \(\widehat{H}\) to determine the optimal neighborhood size, and the maximum neighborhood size, \(M\). The initial step, computing all pairwise distances for data instances in \(\mathbf{X}\), remains the same as in the standard algorithm. Next, the algorithm performs the entropy-based neighborhood selection (lines 3-9). For each point \(x_{i}\), the algorithm starts with an initial neighborhood size, \(k\), and identifies the \(k\)-nearest neighbors. These are used to compute the neighborhood entropy for the data instance. If the entropy threshold \(\widehat{H}\) is not satisfied, then \(k_{i}\) is incremented (line 6), and the process repeats. Once the entropy threshold is reached, or if \(k_{i}\) reaches the maximum threshold of \(M\), the process terminates. The entire process is repeated for each \(x_{i}\), and after all neighborhoods have been identified, the algorithm continues the same way as standard Isomap (lines 10-12). While this iterative strategy for identifying optimal neighborhood size for each data instance appears to be inefficient, it is presented here for more clarity. In practice, the optimal neighborhood determination can be made via a more efficient binary search-based strategy.