# Trident: A Deep Learning Framework for High-Resolution Bridge Vibration Monitoring
## 2 Trident  
CNNs and recurrent neural networks (RNNs) have existed for several decades [24; 25]. CNNs are one of the backbones of deep learning models developed for computer vision, and RNNs played a significant role in natural language processing [26]. The convolution operator helps to extract features with great computational efficiency compared with fully connected neural network layers. This is a critical advantage for problems that deal with high-dimensional input-output spaces, such as high-resolution images and, in the case of SHM, structures that can be modeled inside a large 3D grid of nodes [27]. Despite this advantage, CNNs may not be effective in learning temporal dependencies in sequential data like time series. Variants of RNNs are common in such circumstances, and have a long history of applications in natural language processing [28] and automatic speech recognition [29]. The disadvantage of RNNs is that their ability to process very long sequences is limited. Furthermore, handling multiple time series using a single model with end-to-end training could be highly restricted by the availability of computational resources. An example of this phenomenon in the SHM domain is monitoring acceleration histories from a dense array of sensors installed on a building or a bridge. [PERSON] et al. [30] propose the ConvLSTM operator that treats the input data as a spatiotemporal sequence. In simple words, this architecture marries the computational advantages of CNNs with RNN's capabilities in learning from sequential data. The authors replace fully connected layers in a traditional Long short-term memory (LSTM) cell with convolution operators that assist in input-to-state and state-to-state transitions. It is critical to identify the dimensionality of the vibration input tensor to understand how the ConvLSTM cells operate in Trident. Each node in a structure can be assumed as a part of a 3D grid. This grid is essential as it helps the deep learning model receive an input tensor sensitive to nodes' relative X, Y, and Z coordinates. The original implementation is developed for precipitation nowcasting and utilizes a 2D geometric grid and, therefore, leverages ConvLSTM2D [30]. However, solid objects, such as structures, are 3D, which justifies the need for ConvLSTM3D cells. In addition to the physical geometry, each node can contain several channels that record different data streams. For example, a triaxial accelerometer records vibration response in three directions. Moreover, SHM might involve other response types, such as strains which could add to the total number of channels (\(n_{c}\)). In this study, inspired by [PERSON] and [PERSON] [22], Mel Filter Banks (MFBs) are extracted from acceleration time series to capture variations in time and frequency domains. The first dimension (horizontal axis) of an MFB tensor corresponds to overlapping time windows, and the second dimension contains filter values that indicate the amount of energy stored in different frequency ranges of each time window. With that said, \(n_{t}\) time frames and \(n_{f}\) filter banks add two dimensions to the input tensor. As a result, a structure's vibration input data can be considered a six-dimensional (6D) tensor of shape (\(n_{x}\), \(n_{y}\), \(n_{z}\), \(n_{c}\), \(n_{t}\), \(n_{f}\)). We only consider acceleration response for sensor channels (\(n_{c}=3\)) because this type of sensing is more common for vibration-based civil infrastructure monitoring. This concept can be extended to scenarios where other types of sensors (e.g., strain gauges) are utilized. For this reason, Trident (as its name implies) is an architecture with three branches. Each branch receives a 5D tensor of shape (\(n_{x}\), \(n_{y}\), \(n_{z}\), \(n_{t}\), \(n_{f}\)), which fits a ConvLSTM3D cell that enables learning from temporal variations of vibrations in a 3D space.