# Data and Knowledge-Driven Bridge Digital Twin Modeling for Smart Operation and Maintenance
## 4 Spatiotemporal Data Collection and Processing for Bridge Digital Twin
### Time Series Sensory Data Collection and Processing
#### 4.3.2 Sensor Data Preprocessing  
Preprocessing time series data is an essential step in DTM as it ensures that raw sensor data are preprocessed, reliable, and suitable for advanced analysis. Given that sensor outputs often contain noise, missing records, and inconsistencies, it is necessary to address these issues to enhance the data's usability for predictive models, such as LSTM networks. Therefore, the key preprocessing tasks include noise reduction, data normalization, imputation of missing data, and transforming the data into a format appropriate for modelinput. These steps collectively ensure data integrity and accuracy, allowing for meaningful insights into the structure's behavior over time.  
The first step in preprocessing is noise reduction as raw data often contain unwanted variations that can obscure the true structural signals. Noise typically arises from environmental factors, equipment calibration errors, or mechanical disturbances. To mitigate these, one of the simplest techniques is a moving average filter [47]. For instance, the smoothed value \(\hat{x}_{t}\) at time \(t\) can be computed over a window \(w\) as in Formula (12):  
\[\hat{x}_{t}=\frac{1}{w}\sum_{i=t-w+1}^{t}x_{i} \tag{12}\]  
where \(\hat{x}_{t}\) represents the smoothed value at time \(t\), \(x_{i}\) represents the observed value at time \(i\), and \(w\) is the size of the smoothing window. This step smooths out short-term fluctuations.  
Once noise has been reduced, the next step is data normalization. Since various sensors measure different physical quantities on different scales, normalization aligns all sensor outputs to a common range. This is crucial, especially when feeding the data into machine learning algorithms, which expect features to be on comparable scales. A commonly used normalization method is min-max normalization, which transforms the data into a specified range, typically [0, 1], as in Formula (13):  
\[{x_{t}}^{\prime}=\frac{x_{t}-min(x)}{max(x)-min(x)} \tag{13}\]  
where \(x_{t}\) is the normalized value at time \(t\), and \(min(x)\) and \(max(x)\) are the minimum and maximum values in the dataset, respectively.  
Following normalization, it is important to handle missing data, which often occur due to sensor malfunctions, transmission errors, or environmental factors. Ignoring missing values could distort the overall analysis; hence, imputation techniques are employed to fill in the gaps. A basic approach is linear interpolation, which estimates missing values based on neighboring observations. If data are missing at timestamp \(t_{m}\), the imputed value \(x_{t_{m}}\) is computed as in Formula (14):  
\[x_{t_{m}}=x_{t_{m-1}}+\frac{t_{m}-t_{m-1}}{t_{m+1}-t_{m-1}}(x_{t_{m+1}}-x_{t_{ m-1}}) \tag{14}\]  
Figure 7: Time series sensory data collection and processing.  
where \(t_{m-1}\) and \(t_{m+1}\) are the timestamps surrounding \(t_{m}\), and \(x_{t_{m-1}}\) and \(x_{t_{m+1}}\) are the observed values at those timestamps.  
Once the data gaps are addressed, the next step is data transformation, which enriches the time series by extracting features and reducing dimensionality. Transformations like Fourier transforms convert time-domain data into the frequency domain, making it easier to detect periodic patterns that may not be obvious in the original sensor readings [48]. The discrete Fourier transform (DFT) of a time series \(x_{t}\) is defined as in Formula (15):  
\[X_{k}=\sum_{t=0}^{N-1}x_{t}e^{-2\pi i\mathrm{kt}/N} \tag{15}\]  
where \(X_{k}\) is the frequency component at index \(k\), \(x_{t}\) is the value of the time series at time \(t\), and \(N\) is the total number of time points in the series.  
At this stage, the preprocessed data can be used as input for models such as LSTM, which are designed to capture temporal dependencies and predict future structural health. By following these rigorous preprocessing steps, the BDT model can more effectively detect early signs of degradation, enabling timely interventions and maintenance.