## 4 Entropy-Isomap
```
0:\(\mathbf{X},k\), \(\widehat{H}\), \(M\)
0:\(\mathbf{Y}\)
1:\(\mathbf{D}_{n\times n}\leftarrow\textsc{PairwiseDistances}(X)\)
2:\(\mathbf{G}_{n\times n}\leftarrow\infty\)
3:for all\(x_{i}\in\mathbf{X}\)do
4:\(k_{i}\gets k\)
5:while\(H<\widehat{H}\)and\(k_{i}<(M+k)\)do
6:\(k_{i}\gets k_{i}+1\)
7:\(\mathbf{kNN}\leftarrow\textsc{KNN}(x_{i},\mathbf{X},k_{i})\)
8:\(\mathbf{G}_{i,j}\leftarrow\mathbf{D}_{i,j}\) where \(x_{j}\in\mathbf{kNN}\). 9:\(H\leftarrow\textsc{NeighborhoodEntropy}(x,k_{i},\vec{G}_{i})\)
10:\(\mathbf{F}_{n\times n}\leftarrow\textsc{AllPairsShortestPaths}(\mathbf{G})\)
11:\(\mathbf{Y}\leftarrow\textsc{MDS}(\mathbf{F})\)
12:return\(\mathbf{Y}\)
```  
**Algorithm 2**Entropy-Isomap  
We applied Entropy-Isomap to the process data discussed earlier, with \(k=8\) and the maximum number of steps \(M=100\). The large \(M\) ensures that the algorithm is able to create very large neighborhoods to strictly enforce trajectory mixing. To understand the behavior of the method as a function of the entropy threshold, \(\widehat{H}\), we varied \(\widehat{H}\) from 0.1 to 0.9. The example low-dimensional representation obtained by Entropy-Isomap is presented in Figure 6b. Figure 5: Neighborhood entropy of different trajectories as a function of \(k\) (\(\chi=3.0\) and \(\phi\in\{0.50,0.52,0.54,0.56,0.58,0.6\}\)). Figure 7b shows how the final neighborhood entropy for the instances belonging to six trajectories vary according to time. We note that, for this data, the threshold for entropy (\(\widehat{H}\), set to 0.3 for this experiment) is often not reached even after expanding \(k_{i}\). Instead, the neighborhood size reaches the maximum limit, \(M\). We believe that this is because the \(k\) nearest neighbors for the majority of points are in the same trajectory, (see Figure 4b), which leads to skewed neighborhood distributions. As a result, even when a satisfactory number of neighbors come from other trajectories, the entropy for the neighborhood might be low. Even when the trajectories mix, the neighborhoods are still dominated by other instances in the same trajectory, as shown in Figure 6c. Therefore, while high entropy implies good mixing, the converse is not necessarily true. Large neighborhoods could produce mixing, while still having low entropy. Thus, the entropy threshold is set to be around 0.30-0.40, with further confirmation through experimentation being presented in the next section. If the entropy threshold is strictly enforced, one will have to increase the neighborhood size. For the process data, we observed that the neighborhood sizes will become very large. Figure 7a shows the neighborhood size distribution for \(\widehat{H}=0.30\). For such values of \(k\), the method simply reduces to PCA, a linear method, which suffers from limited expressiveness because of the linearity assumption. Points that produce no mixing also end up with large neighborhoods, as Entropy-Isomap tries to increase \(k_{i}\) in order to meet the entropy threshold. These points occur when the dataset does not  
Figure 6: Six trajectories with fixed \(\chi=3.0\) and the variable \(\phi\in\{0.50,0.52,0.54,0.56,0.58,0.6\}\) were selected to learn mapping and transform the data into three dimensions using **(a)** Isomap with \(k=8\) and **(b)** Entropy-Isomap with \(k=8\), \(\widehat{H}=0.3\). **(c)** Neighborhood cross-mixing given by Entropy-Isomap with \(k=8\), \(\widehat{H}=0.3\): for each trajectory \(\Gamma\), neighbors of each point belonging to individual trajectories are aggregated and shown in stacked bar graph form. (Please view in color). Figure 7: Entropy-Isomap with the default \(k=8\) and entropy threshold \(\widehat{H}=0.3\) run for data with \(\chi=3.0\) and the variables \(\phi\in\{0.50,0.52,0.54,0.56,0.58,0.6\}\). (Please view in color). (**a)** Distribution of selected neighborhood sizes \(k_{i}\) that did not reach the maximum \(M=100\). (**b)** Entropy of the discovered neighborhood for each time step.