# Building Survivable Software Systems by Automatically Adapting to Sensor Changes
## 4 Empirical Study
### Results on Weather Data  
We used 30 weather stations from 10 geographical clusters. We generated random triplets across clusters. We generated each triplet in the following way: (1) randomly select two clusters; (2) randomly select two stations from the first cluster (denoted as the stations A1 and A2), and one station from the second cluster (denoted as the station B). We assumed that sensors in B were correlated with those in A1 and A2. We used sensors in A1 as the compound sensor, sensors in A2 as the new sensors, and sensors in B as the reference sensors. We generated 100 random triplets, and we report results averaged on them. Each station consisted of six sensors including temperature (\({}^{\circ}\)F), humidity (%), dew point (\({}^{\circ}\)F), wind speed (mph), wind gust (mph), and pressure (Pa). Sensor values were collected every 5 min and are temporally aligned. Since A1 and A2 were from the same cluster, sensor values from A1 and A2 were typically more correlated than those from A1 (A2) and B. We used 2 years of data, with data in 2016 as the source domain and data in 2017 as the target domain. Individual Sensor Changes: we treated each sensor in A1 as the replaced sensor, the remaining sensors in A1 plus all sensors in B as the reference sensors, and all sensors in A2 as the new sensors. Table 1 reports average reconstruction errors and the corresponding standard errors, with Imp. showing the average improvement (in %) of ASC over the best  
Figure 4: UUV Sensors (RPM, Waterspeed, DVL). baseline method. We can see that ASC achieved an average improvement of 6.4% over baselines. This showed the high robustness of ASC. In general, ASC showed more statistically significant improvement on sensors whose values exhibited large variances (e.g., wind gust and pressure). Replace always underperformed compared to Refer, revealing that directly using new sensors could cause significant differences in sensor values. ReferZ performed better than Refer by leveraging new sensors. ASC further improved over ReferZ because it better exploited information from new sensors. Figure 5 visualizes the joint distributions over wind speed and reconstructed pressure on a station in San Francisco (x-axis: wind speed, y-axis: reconstructed pressure). Figure 5a is the ground-truth distribution that we would like to approximate after adaptation. As we can observe, Refer generated a significantly different joint distribution compared to the ground truth, while ASC produced a much closer distribution by leveraging new sensors. Compound Sensor Changes: we treated all sensors in A1 as the replaced sensors, all sensors in B as the reference sensors, and all sensors in A2 as the new sensors. Table 2 reports reconstruction errors on each replaced sensor separately. ASC statistically outperformed baselines in three cases, achieving an average improvement of 5.7%. Compared to Table 1, ASC produced larger reconstruction errors mainly because the reference sensors had lower correlations with the replaced sensors in this case. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Sensor** & **Replace** & **Refer** & **ReferZ** & **ASC** & **Imp.** \\ \hline temperature & 3.94 \(\pm\) 0.024 & 0.61 \(\pm\) 0.011 & 0.59 \(\pm\) 0.009 & **0.57 \(\pm\) 0.010** & 4.1 \\ humidity & 5.73 \(\pm\) 0.023 & **0.72 \(\pm\) 0.016** & **0.71 \(\pm\) 0.015** & **0.72 \(\pm\) 0.015** & \(-\)1.7 \\ dew point & 3.89 \(\pm\) 0.027 & 0.70 \(\pm\) 0.010 & **0.68 \(\pm\) 0.009** & **0.67 \(\pm\) 0.010** & 2.8 \\ wind speed & 8.24 \(\pm\) 0.084 & **5.20 \(\pm\) 0.063** & **5.21 \(\pm\) 0.064** & **5.11 \(\pm\) 0.060** & 1.7 \\ wind gust & 10.81 \(\pm\) 0.073 & 6.65 \(\pm\) 0.052 & 6.65 \(\pm\) 0.048 & **6.31 \(\pm\) 0.046** & 5.0 \\ pressure & 7.82 \(\pm\) 0.16 & 3.42 \(\pm\) 0.19 & 2.48 \(\pm\) 0.17 & **1.83 \(\pm\) 0.17** & 26.2 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Reconstruction errors (RMSE) on weather data for individual sensor changes. Each entry shows the average reconstruction error and the corresponding standard error. Imp.