## 3 Challenges of Using SDR with Dynamic Process Data
### Comparing PCA and Isomap for Dynamic Process Data  
A reliable way of determining the quality of the low-dimensional representation (mapping) produced by each method is to compare the original data \(\mathbf{X}\) in \(\mathbb{R}^{D}\) with the mapped data \(\mathbf{Y}\) in \(\mathbb{R}^{d}\), by computing the _residual variance_. The process of computing residual variance for PCA differs from Isomap, but the values are directly comparable.  
In PCA, each principal component (PC) explains a fraction of the total variance in the dataset. If we consider \(\lambda_{i}\) as the eigenvalue corresponding to the \(i\)th PC and \(|\Lambda|\) as the total energy in the spectrum, i.e., \(|\Lambda|=\sum_{i=1}^{D}\lambda_{i}\), then the variance explained by the \(i\)th PC can be computed as \(\frac{\lambda_{i}}{|\Lambda|}\). The residual variance can be calculated as  
\[R=1-\sum_{i=1}^{d}\frac{\lambda_{i}}{|\Lambda|}. \tag{1}\]  
In the Isomap setting, residual variance is computed by comparing the approximate pairwise geodesic distances, computed in \(G\) represented by matrix \(\mathbf{D}_{G}\) (recall that \(G\) is a neighborhood graph), to the pairwise distances of the mapped data \(\mathbf{Y}\), represented by matrix \(\mathbf{D}_{Y}\):  
\[R=1-\rho(\mathbf{D}_{G},\mathbf{D}_{Y})^{2}. \tag{2}\]  
Here, \(\rho\) is the standard linear correlation coefficient, taken over all entries of \(\mathbf{D}_{G}\) and \(\mathbf{D}_{Y}\).  
To compare PCA and Isomap, we compare the residual variance obtained using PCA and Isomap on the earlier described material morphology evolution process data. The dataset consisted of six trajectories, corresponding to a unique combination of the parameters \(\phi\) and \(\chi\). The comparison is shown in Figure 2. It is evident that PCA is not able to learn a reasonable low-dimensional mapping even when using 10 eigenvectors, while Isomap produces a highly accurate representation. For instance, while Isomap is able to explain about 70% of the variance using three dimensions, PCA requires more than nine dimensions. It should be noted that the ability to represent data in two or three dimensions is especially desired by domain experts, as it allows for data visualization and exploratory analysis.  
Figure 2: Isomap and principal component analysis (PCA) run on simulation output produced by data with six trajectories for \(\chi=3.0\) and \(\phi\in\{0.50,0.52,0.54,0.56,0.58,0.6\}\). The quality of the Isomap manifold and PCA subspace are assessed using residual variance.  
If we visualize the data in 3D space using the PCA representation (\(d=3\)) (see Figure 2(a)), we observe that while PCA is able to describe the time aspect of the process evolution, it does not offer any additional insights into the process, making it ineffective for the task at hand. This can be primarily attributed to PCA's inability to capture the nonlinear relationships between the high- and low-dimensional data.  
The 3D visualization of Isomap is shown in Figure 2(b). We note that despite the fact that Isomap is better than PCA at minimizing the residual variance, the low-dimensional trajectories do not offer meaningful insights. The trajectories appear to diverge from one another, leaving no reasonable interpretation of the empty space, whereas one would expect some ordering with respect to the \(\phi\) parameter. This indicates that the standard application of Isomap is inadequate when working with parameterized high-dimensional time series data. We obtain equally unsatisfactory results with other methods, including t-SNE and LLE [14; 25].