# Data and Knowledge-Driven Bridge Digital Twin Modeling for Smart Operation and Maintenance
## 4 Spatiotemporal Data Collection and Processing for Bridge Digital Twin
### The 3D Point Cloud Data Collection and Processing
#### 4.2.3 The 3D Point Cloud Processing Using PointNet  
Once the 3D point cloud data have been preprocessed, they are further processed with the PointNet model, a neural network specifically designed for handling unstructured point cloud data [42]. PointNet operates by applying multi-layer perceptrons (MLPs) to each point individually, allowing for the extraction of both local and global features from the data. This unique capability makes PointNet particularly effective in identifying and quantifying geometric deformations in bridge structures, as illustrated in Figure 6.  
Figure 6: Schematic representation of the PointNet model.  
The process begins with local feature extraction for each point \(p_{i}=(x_{i},y_{i},z_{i})\). PointNet applies a sequence of MLP layers, where each layer transforms the point's feature representation. The transformation at the \(k\)-th layer is governed by Formula (8):  
\[h_{i}^{(k)}=\sigma\left(W_{k}h_{i}^{(k-1)},+b_{k}\right) \tag{8}\]  
where \(h_{i}^{(k)}\) represents the feature vector after the \(k\)-th layer, \(W_{k}\) is the weight matrix of the \(k\)-th layer, \(b_{k}\) is the bias term at the same layer, and \(\sigma\) is the non-linear activation function applied to the output.  
This process generates a high-dimensional feature vector for each point, effectively capturing local geometric characteristics that are critical for monitoring structural health.  
Next, global feature aggregation is performed to summarize the entire structure's geometric shape. This is achieved using a max-pooling operation, which extracts the most prominent features from all the points. The global feature vector \(g\) is defined as in Formula (9):  
\[g=\frac{N}{\underset{i=1}{\max}\{h_{i}\}} \tag{9}\]  
where \(\sigma\) is the global feature vector that captures the overall geometry, \(N\) is the total number of points in the point cloud, and \(h_{i}\) represents the local feature vector of the \(i\)-th point.  
This aggregation ensures that the model captures the bridge's overall structural properties, regardless of the number of points, by focusing on the most significant features across the dataset.  
For structural deformation monitoring, PointNet enables the calculation of point displacements over time. By comparing the current and reference point clouds, the displacement of each point is calculated as in Formula (10):  
\[d_{i}=\|p_{i}^{current}-p_{i}^{reference}\| \tag{10}\]  
where \(d_{i}\) is the Euclidean distance representing the displacement of point \(i\), \(p_{i}^{current}\) is the position of point \(i\) in the current point cloud, and \(p_{i}^{reference}\) is the position of point \(i\) in the reference point cloud.  
By analyzing the computed displacements, localized structural deformations, such as cracks or shifts, can be identified, providing valuable insights into the integrity of the bridge.  
Finally, component classification and segmentation allow for the differentiation between various structural elements of the bridge, such as piers, decks, and girders. The classification process aims to minimize the cross-entropy loss as in Formula (11):  
\[\mathcal{L}_{class}=-\sum_{i=1}^{N}y_{i}\log\hat{y_{i}} \tag{11}\]  
where \(\mathcal{L}_{class}\) is the cross-entropy loss used to evaluate the classification performance, \(y_{i}\) represents the true class label of point \(i\), and \(y_{i}\) is the predicted probability of point \(i\) belonging to a particular class.  
This segmentation enables the system to monitor specific components of the bridge in a more targeted manner, allowing for early detection of issues in distinct areas, such as cracks in the piers or deformation in the girders.  
By leveraging PointNet's ability to process unstructured point clouds, this approach supports real-time monitoring of structural deformations. This capability is crucial for providing early warnings in bridge health management systems.