# Building Survivable Software Systems by Automatically Adapting to Sensor Changes
## 4 Empirical Study  
We evaluated ASC on sensor data from the weather and UUV domains.  
* Weather Data: the dataset consisted of weather sensor data collected from Weather Underground ([[https://www.wunderground.com/](https://www.wunderground.com/)]([https://www.wunderground.com/](https://www.wunderground.com/)), accessed on 17 May 2021). The dataset involved a number of personal weather stations; and each station (compound sensor) contains a set of individual sensors including temperature, dew point, humidity, wind speed, wind gust, pressure, etc. These weather stations were selected from a set of clusters/regions (e.g., Los Angeles, San Francisco, Austin, Chicago, etc.), each with three stations. The clusters were determined based on the cities in which the stations are located. Stations within a cluster tended to produce more similar sensor values than those across clusters. Sensor values were sampled every 5 or 10 min. We temporally aligned sensor values as a preprocessing step. The data used in our empirical study are available for download ([[https://github.com/usc-isi-2/sensor-adaptation/tree/master/datasets/weather/dataset](https://github.com/usc-isi-2/sensor-adaptation/tree/master/datasets/weather/dataset)]([https://github.com/usc-isi-2/sensor-adaptation/tree/master/datasets/weather/dataset](https://github.com/usc-isi-2/sensor-adaptation/tree/master/datasets/weather/dataset)), accessed on 17 May 2021). The data were organized into different geographical clusters where each cluster contains a few weather stations. Sensor values were represented in JSON format where each attribute corresponded to a different sensor type.
* UUV Data: the dataset was collected by letting a UUV travel from a starting point to an end point in a simulated environment. The UUV contained a propeller RPM sensor, a watershed sensor, and a compound sensor called Doppler Velocity Log(DVL) sensor. The DVL sensor consisted of seven individual sensors including surge, heave, sway, pitch, roll, depth, and heading. Figure 4 shows the locations of these sensors on a UUV. Each sensor produced a sensor value every second. We simulated 20 trips and collected sensor values at each second. The trajectory of the UUV varies in each trip due to different starting/end points and water currents. The total number of samples in each trip varied between 500 and 2000.  
We compared ASC to three baseline methods:  
* Replace: non-adaptation method that substitutes each replaced sensor with a new sensor that has the closest mean and variance in sensor values.
* Refer: adaptation method that reconstructs sensor values of replaced sensors using reference sensors, without exploiting any new sensor.
* ReferZ: adaptation method that works in the following three steps: 1. Learn a regression model on the target domain to reconstruct the new sensors from reference sensors. 2. Use the learned regression model to reconstruct the new sensors on the source domain. 3. Learn a reconstruction function on the source domain to reconstruct replaced sensors from reference sensors and reconstructed new sensors.  
This method could work well if the new sensors and reference sensors were strongly correlated, which may not hold in real-world applications.  
The reconstruction error of each method was measured by RMSE (Root Mean Square Error) between the reconstructed sensor values and the ground truth. To show the robustness of the reconstruction errors, we also report the corresponding standard errors.