Drive-by Health Monitoring utilizes accelerometers mounted on vehicles to gather dynamic response data thatcanbeusedtocontinuouslyevaluatethehealthofbridgesfasterandwithlessequipmentthantraditional structural health monitoring practices. Because vehicles and bridges create a coupled system, vehicle accel- eration data contains information about bridge frequencies that can be used as health indicators. However, for drive-by health monitoring to be viable, variabilities in dynamic measurements caused by environmental and operational parameters, such as temperature, vehicle speed, traﬃc, and surface roughness need to be considered. In this paper, a ﬁnite element model of a simply supported bridge is developed considering the aforementioned variabilities and various levels of structural damage. Vehicle acceleration data obtained from the model is analyzed in the frequency domain and processed using a neural network architecture. This method is used to determine the relationships between noise inducing variables and changes in vehicle dynamic response spectrum; these relationships are leveraged to predict the overall health of the subject bridge. The results from this study indicate that the proposed approach can serve as a viable health moni- toring strategy and should be further tested on physical bridge systems. Reproducibility: our code and data are available at [link upon acceptance] Keywords: Structural Health Monitoring, Drive-by Health Monitoring, Finite Element Models, Neural Net, Highway Bridge, Transportation Infrastructure. 2010 MSC: 00-01, 99-00 1. Introduction Maintaining transportation networks is essential to supporting national security, the economy, and public safety. Because of delayed maintenance and budget limitations, however, bridges that are critical to the U.S. transportation network are in a state of disrepair [1, 2]. Per the 2017 American Society of Civil Engineering report, approximately 240 thousand bridges in the United States have exceeded their 50-year service life, and over 56 thousand bridges have been classiﬁed as structurally deﬁcient [1]. The situation in other countries is not much diﬀerent [3]. Further delays in the maintenance of this vital infrastructure system poses a signiﬁcant risk of bridge failures, which would threaten national security, cause further economic losses, and result in the loss of lives. As such, a more eﬃcient and cost eﬀective health monitoring and maintenance strategy would aid to improve the state of this failing system by optimizing the allocation of funds and resources to where they are most needed. Corresponding author Email addresses: wrlocke@clemson.edu (William Locke), jsybran@clemson.edu (Justin Sybrandt), lmredmo@clemson.edu (Laura Redmond), isafro@clemson.edu (Ilya Safro), sez@psu.edu (Sez Atamturktur) 1Glenn Department of Civil Engineering, Clemson University 2School of Computing, Clemson University 3Department of Architectural Engineering, Pennsylvania State University Preprint submitted to Journal of Sound and Vibration July 19, 2019 Presently, time interval-based visual inspections are the accepted practice for monitoring bridge health in the United States. The National Bridge Inspection Policy regulates the procedures for these inspections and dictates that bridge inspections should be performed every two years [4]. Despite having been criticized for being both ineﬃcient and ineﬀective, this policy has continued to be practiced nation wide. Firstly, it has been argued that using ﬁnancial resources to inspect bridges less than ten years old is ineﬃcient as young bridges often exhibit little degradation [5]. Additionally, lack of ﬁnancial resources may cause owners to miss necessary inspections, with approximately 17000 bridges missing their deadline in 2006, of which approximately 2700 bridges were considered obsolete or deﬁcient [6, 7]. Secondly, visual inspections are ineﬀective as they not only fail to detect certain types of damage, but also rely on subjective assessments that often lead to inconsistent, unrepeatable observations [5, 8, 9, 10]. Structural Health Monitoring (SHM) is an alternative approach that oﬀers a solution to these issues. SHM is the process of implementing a damage detection and characterization strategy for engineered systems. SHM has been adopted as an accepted practice in many engineering disciplines, and is gaining popularity for use in civil engineering [11]. The inclusion of an SHM system as part of an asset monitoring strategy allows for more informed life-cycle management decisions that can help extend the asset’s service life [12, 13, 14]. In bridge health monitoring, an eﬀective strategy typically uses multiple online sensors and intelligent diagnostics to continuously detect damage on a local and/or global scale [11, 15]. However, these typesofSHMsystemsarenotfeasibleforshorttomid-spanbridgesbecauseofassociatedcosts[14,16,17,18]. For example, the initial cost of installing a system on every short to mid-span bridge in the U.S. is estimated to be in the range of tens of billions of dollars [19]. The methodology is made more expensive by essential sensor maintenance that requires signiﬁcant time and resources for site visits [17, 18]. To overcome these limitations, a mobile health monitoring strategy known as Drive-by Health Monitoring (DBHM) has been proposed [14]. DBHM, or indirect health monitoring as it has also been called, is the practice of using accelerometers attached to passing vehicles to detect damage induced changes in the dynamic response of bridges [14]. Because vehicles and bridges create a coupled system, vehicle acceleration data contains information about bridge frequencies that can be used as health indicators. This concept was ﬁrst demonstrated by Yang et al. when they successfully detected bridge frequencies contained within vehicle acceleration data generated from numericalﬁniteelement(FE)simulations[17]. LinandYanglatervalidatedtheconceptexperimentallywhen they successfully detected frequencies contained within the acceleration response of a cart as it was towed across the Da-Wu-Lun bridge in Taiwan [20]. The goal of this strategy is to be able to continuously monitor bridges in a more eﬃcient, cost eﬀective, and less labor-intensive manner than any other forms of SHM or inspection practice; however, before this goal can be obtained, variabilities introduced by operational and environmental factors need to be addressed to ensure the accuracy of the health assessment being reported. Operational and environmental parameters, such as bridge deck surface roughness, vehicle speed, traf- ﬁc, and temperature, have been known to mask or shift bridge frequencies in such a way that accurately classifying damage becomes very diﬃcult [20, 21, 22]. Progressively rougher surface proﬁles introduce high amplitude noise that obscures the process of extracting bridge frequencies from a vehicle’s dynamic response. Correspondingly, at higher vehicle speeds, surface roughness introduces high frequency vibrations that fur- ther complicate bridge frequency detection [20, 22]. Higher vehicle speeds also cause the spectral resolution of sensors to decrease and the extracted frequencies to shift away from their actual values [17, 20]. On-going traﬃc has been demonstrated to actually aid in the drive-by health monitoring paradigm by increasing the acceleration amplitude associated with extracted bridge frequencies [20, 22], but others note that traﬃc can also cause these frequencies to ﬂuctuate by several percent as a result of vehicle masses altering the eﬀective bridge mass [23, 24, 25]. Temperature has a similar eﬀect, causing frequencies to ﬂuctuate by several percent over a 24-hour period, and by an even larger magnitude throughout the year [21, 26, 27]. Researchers have been able to address these issues by using an assortment of signal processing, statistical, and machine learning techniques [14]. These solutions, however, do not consider the cumulative eﬀects of the aforementioned parameters and, therefore, cannot account for all the variation observed in real-world frequency data, which can potentially cause misidentifying damage. In this paper, we propose a method for assessing bridge health, while considering the cumulative eﬀects of various operational and environmental factors. 2 To assess bridge health using DBHM while considering cumulative environmental and operational eﬀects, we propose an approach that ﬁrst trains a neural network (NN) to detect damage on realistic vehicle-bridge simulations, andthenappliesthetrainednetworktowardsdetectingsimilardamageonphysicalsystems. The novelty of this methodology is based on the development of a new approach to drive-by health monitoring that relies on detecting physical damage through the development of simpliﬁed yet realistic vehicle-bridge simulations. To evaluate the feasibility of this approach, however, a study ﬁrst needs to be performed to determine if an actionable health assessment can reliably be obtained from simulations of vehicle-bridge systemsexposedtonoise. Therefore,thenoveltyofthisworkisthedevelopmentofvehicle-bridgesimulations, which contain realistic signal noise generated from temperature changes, road surface roughness, and other vehicle traﬃc, to train and test the ability of a NN to make a reliable health assessment. Additionally, to ensure the levels of damage implemented in the simulation are representative of possible damage states to a physical system, we implement a unique approach for relating levels of crack damage to changes in moment of inertia (MoI) and, therefore, principle bridge frequency. In this study, we utilize a ﬁnite element methodology that represents the interaction between vehicles and a bridge as a contact force to simulate a simply supported single-span bridge coupled with moving quarter-vehicle models [28]. A variety of diﬀerent vehicles are simulated to represent daily traﬃc ﬂow; the number of diﬀerent vehicles in this study represent a hypothetical scenario in which almost all vehicles crossing a bridge are outﬁtted with sensors to allow for the continuous monitoring of bridge health. Realistic representations of environmental and operational variabilities, along with bridge damage, are simulated by the model. To represent long-term natural variations in bridge frequency, we evaluate our model across approximately two years of operational and environmental observations. Then, we leverage a convolutional NN to capture a noise-tolerant relationship between sensor observations and bridge health. Input to the NN model is projected across a number of layers, each capturing an increasingly abstract representation of our input (e.g. frequencies, vehicle speed, temperature, etc.) when trained using stochastic gradient decent [29]. These layers are then projected down into a small set of discrete “damage classes,” each representing a diﬀerent level of bridge health. In order to explore the feasibility of this NN approach, sixteen diﬀerent simulation scenarios are evalu- ated. Each represents a unique combination of parameters, including the number of vehicles simultaneously presentonthebridge, whetherenvironmentaleﬀectsareaccountedfor, whetherbridgedamageislocalizedor expanding, and what weather data underpins the simulated temperature. For each scenario independently, vehicle crossings are recorded as a series of measurements. These include frequency-amplitude pairs corre- sponding to the upper and lower masses of the quarter-vehicle model, derived from Fast Fourier Transforms (FFTs) applied to the vertical acceleration of each mass during the crossing. In addition, measurements of time-of-day, temperature, estimated vehicle mass and speed are all associated with each crossing. This crossing data is randomly shuﬄed and split into training, validation, and test sets for the purpose of con- structing and evaluating the NN model. The NN model then learns to associate patterns present in these measurements with each of the damage classes. The test-set evaluate is comprised of two main metrics, accu- racy, and “conﬁdence”. Accuracy indicates how often a single vehicle correctly predicts the bridge’s damage class given a single crossing. Conﬁdence, a custom metric, pools the predictions of all test-set vehicles that cross the bridge on the same simulated day, and assigns the bridge a damage class by a majority vote of the vehicle predictions. Then, conﬁdence indicates the fraction of vehicles that voted in the majority, and whether that majority was correct. Semantically, accuracy indicates generalized NN performance across all vehicles and damage classes, while conﬁdence is plotted in order to depict what damage classes or times of year are more challenging to predict. This paper is organized as follows: Section 2 discusses the applied ﬁnite element method and includes parameter values for the bridge and vehicle models. Section 3 addresses simulating operational, environmen- tal, and damage eﬀects on the subject bridge; Section 4 discusses the NN and how it was applied towards the subject data; Section 5 discusses the outputs from the NN and summarizes the results; Section 6 summarizes the ﬁndings of the study; and Section 7 provides suggestions for future research. 3 Figure 1: Quarter-vehicle model coupled with simply-supported bridge. 2. Finite Element Method for Vehicle-Bridge Interaction 2.1. FEM In this study, vehicle-bridge interactions are modeled using a two dimensional ﬁnite element simulation based on the concept of vehicle-bridge interaction elements [28]; see Figure 1 for an example vehicle-bridge system. Because the majority of bridges in the United States are short-spanned (i.e. less than 50m (160ft) ), and because most steel and concrete bridge designs can be employed using simply supported beams, the bridge in this study is modeled as a short-spanned simply supported beam that is discretized into a number of Euler-Bernoulli beam elements [30, 31, 32, 33]; the number of elements the beam is discretized into depends on the damage condition being applied, see Section 3.3 for more details. Equation 1 represents the equation of motion for an occupied beam element in the future time step ( t+4t): ([mbi]fdbig+ [cbi]f_dbig+ [kbi]fdbig)t+4t= (ffbigffbcig)t+4t; (1) where [mbi],[cbi], and [kbi]represent the ithlocal element’s mass, damping, and stiﬀness matrices, respec- tively. The column vectors fdbig,f_dbig, andfdbigrespectively indicate nodal acceleration, velocity, and displacement for the local element. The column vector ffbigcontains nodal external forces, while ffbcigis a column vector containing the nodal forces resulting from contact forces introduced by traversing vehicles. Elemental matrices are pieced together to create the global structural matrices and external load vectors free of any vehicles. For this study, a four degree-of-freedom stiﬀness matrix and a four degree-of-freedom continuous mass matrix are implemented. Axial degrees-of-freedom are ignored because the modal frequen- cies associated with vertical degrees-of-freedom are of primary interest, and because vehicles are assumed to travel at constant velocities (i.e. they do not introduce axial loads from acceleration). Because it is diﬃcult to derive the local damping matrix without having prior knowledge of the systems frequency values, the global damping matrix is derived using the Rayleigh damping method outlined in Eqs. 2 and 3 [34]: [Cb] =b0[Mb] +b1[Kb] (2) b0=2!1!2 !1+!2b1=2 !1+!2; (3) where [Mb],[Cb], and [Kb]represent a bridge’s global mass, damping, and stiﬀness matrices, respectively. The parameters b0andb1are numerical constants calculated using the damping ratio and the ﬁrst two undamped modal frequencies w1andw2. The model frequencies are calculated using the eigen value function eig([Kb];[Mb])in Matlab. Once the global bridge matrices have been formulated, the next step is to calculate the global contact forces; this is done by ﬁrst analyzing the equation of motion of the subject vehicle(s). For the purposes of developing and testing the feasibility of the proposed DBHM approach, each vehicle is modeled as a 4 simple two degree-of-freedom mass spring damper (i.e. quarter-vehicle), as seen in Figure 1. Quarter-vehicle models are considered suitable for this study because they have been shown to accurately capture the vertical dynamic behavior of a motor vehicle suspension, and have been regularly employed, along with single degree- of-freedom models, for testing new concepts in the drive-by health monitoring ﬁeld [14, 17, 31, 35, 36, 37]. It known that the quarter-vehicle model cannot capture certain physical vehicle behavior, such as rotational and torsional dynamics; however, this is not a concern for this study as the primary interest is capturing the vertical dynamics associated with the ﬁrst few modes of the bridge. Equation 4 represents the equation of motion for a vehicle in the future time step ( t+4t): Mv0 0Mwdv dw +CvCv CvCw+Cv_dv _dw +KvKv KvKw+Kvdv dw t+4t= fve fwe +0 fwc t+4t;(4) whereMvandMware the upper vehicle mass and wheel mass, respectively. KvandCvdenote the stiﬀness and damping of the upper suspension system, while KwandCwdenote the stiﬀness and damping proper- ties of the wheel. Vectors fdg,f_dg, andfdgreceptively represent the vehicle’s acceleration, velocity, and displacement. Vector ffegcontains external forces that act on both the vehicle body and wheel, while fwc indicates the vertical contact force acting on the wheel. To calculate the contact force, Newmark Beta nu- merical integration is leveraged to solve for the future acceleration, velocity, and displacement of the upper vehicle body; these values are calculated using Eq. 5-7: dv;t+4t=a04dva1_dv;ta2dv;t (5) _dv;t+4t=_dv;t+a3dv;t+a4dv;t+4t (6) dv;t+4t=dv;t+4dv; (7) where subscript tindicates values for acceleration, velocity, and displacement in the present time step. The change in the upper vehicle displacement between the present and future time step is denoted by 4dv. Constantsa0a4are used to perform the numerical integration; these values, along with a5a7employed later in the FEM process, are calculated in Eq. 8: a0=1 4t2a1=1 4ta2=1 2 1a3= (1 )4t a4= 4t a 5= t a6= 1a7=4t 2 2 ;(8) where (1 4)represent the variation in acceleration during the incremental time step 4t, and (1 2)represents numerical or artiﬁcial damping introduced by discretization in the time domain [38]. In this study, the average acceleration method for Newmark Beta is utilized with a time step of a thousandth of a second (i.e. 4t=:001s). Once the equations for the future values are determined, they are entered into Eq. 4 to solve for the unknown change in upper vehicle body displacement 4dv. The calculated change in displacement is then entered into Eq. 5-7 to solve for the future values. After solving the equation of motion for the upper vehicle body in the future time step, the future contact force of the wheel can be determined by solving the the equation of motion for the wheel. Using Newmark Beta method, the resulting equation for the wheel contact force is obtained: fwc;t+4t= (Mcdw+Cc_dw+Kcdw+Pc)t+4t+Qc;t; (9) whereMc,Cc, andKcare respectively the contact mass, damping, and stiﬀness; while Pc;t+4trepresents the eﬀects of external forces on the vehicle, and Qc;trepresents the eﬀects of the vehicle’s displacement 5 vector at the beginning of the time step. Equations 10-14 indicate how these parameters are calculated: Mc=Mw (10) Cc= (Cw+Cv) + w 1 vCv (11) Kc= (Kw+Kv) + w 1 vKv (12) Pc;t+4t= w 1 vfve;t+4tfwe;t+4t (13) Qc;t= w 1 vqv;tqw;t; (14) where v, w,qv;t, andqw;tarerepresentativeparametersusedtosimplifyEqs.10-14, andarederivedduring the Newmark Beta integration; for more information on how these values are derived, please reference [28]. Equations 15-18 indicate how these parameters are calculated: v=a0Mv+a5Cv+Kv (15) w=Cva5Kv (16) qv;t=Mv(a1_dv;t+a2dv;t) +Cv(a6_dv;t+a7dv;t)Kvdv;t (17) qw;t=Cv(a6_dv;t+a7dv;t) +Kvdv;t: (18) wherea5a7are Newmark Beta constants introduced in Eq. 8. Using Eq. 19-21 below, the contact force in Eq. 9 is reformulated to be in terms of the occupied bridge element’s nodal displacement, velocity, and acceleration: dw=fNgTfdbigt+4t (19) _dw=f_NgTfdbigt+4t+fNgTf_dbigt+4t (20) dw=fNgTfdbigt+4t+f_NgTf_dbigt+4t+fNgTfdbigt+4t; (21) wherefNgis a column vector containing Cubic Hermitian polynomial shape functions (i.e. the same functions used to derive the elemental stiﬀness matrix) [39]. Equation 22 indicates the vector of shape functions: fNg=13x2 b+ 2x3 b;xc(12xb+x2 b); 3x2 b2x3 b;xc(x2 bxb) ; (22) wherexcis a vehicle’s local position on an element, and xbis the local coordinate on an element (i.e. xb=xc l). To transform the contact force to be in terms of the equivalent nodal forces ( ffbcig), the previously reformulated contact force is multiplied by the transpose of the aforementioned shape functions. Equation 23 shows the ﬁnal equation of motion for an occupied bridge element (VBI element) with the derivedffbcigincluded: ([mbi]fdbig+ [cbi]f_dbig+ [kbi]fdbig)t+4t= (ffbig[M c]fdbig[C c]f_dbig[K c]fdbigfP cg)t+4tfQ cgt;(23) where the asterisked matrices and vectors are the parameters calculated in Eq. 10-14 that have been multi- plied by the shape functions in Eq. 22. Having obtained the ﬁnalized equation of motion for the VBI element, the modiﬁed contact matrices are added into the elemental matrices on the left, the local element matrices and vectors are then assembled into the previously assembled global matrices at their respective global coordinates. Newmark Beta method is then utilized to solve for the global acceleration, velocity, and displacement vectors for the future time step (t+4t). The nodal acceleration, velocity, and displacement values for occupied elements are then substituted into Eq. 19-21 to calculate the future values for the wheel(s); the resulting values are then substituted into the vehicle’s equation of motion (Eq. 4) to calculate the future acceleration, velocity, and displacement values for the upper vehicle body. Once the future accelerations, velocities, and displacements are known for the vehicle(s) occupying the bridge, the global and local positions for the vehicle(s) are updated for the next time step and the analysis is performed again. This iterative process is repeated until the vehicle(s) reach the end of the bridge. 6 Figure 2: Demonstrates an example where a typical bridge cross section is transformed into an equivalent square cross section for numerical modeling. (a): Cross section view of the physical simply supported bridge structure. (b): Shows equivalent properties for steel girders that have been transformed into concrete. (c): Indicates equivalent properties of the entire bridge cross section when represented numerically. 2.2. Bridge and Vehicle Properties To obtain structural properties that are indicative of a real world bridge structure, an example cross section is designed using AASHTO LRFD 2012 Design Speciﬁcations; see Figure 2a for an illustration of the cross-section. The cross section has a 12:19m (40ft) wide 203:2mm (8in) thick concrete deck composed of27:6MPa (4000psi) concrete and reinforced with two layers of number ﬁve rebar; the requirements for coverage, spacing, and minimum area are employed for each layer of reinforcement as per the aforementioned speciﬁcations [40]. The reinforced deck rests on top of six W 24x250steel girders spaced at 2:44m (8ft) on center; 76:2mm (3in) thickhaunchesareplacedbetweenthedeckandgirderstoallowfortheeﬃcienttransfer of moment in the composite section [40]. The subject bridge has two 3:66m (12ft) wide lanes, two :61m (2ft) wide shoulders, two 1:2m (4ft) wide sidewalks, and two reinforced concrete barrier rails with a base width of:61m (2ft) . Table 1 indicates the structural properties for the subject bridge. To obtain the elastic modulus, MoI, and area properties outlined in Table 1, the equivalent area method is employed towards the steel girders and reinforcement in the subject cross section to create an equivalent cross section of concrete, see Figure 2b for an example. Only structural members are considered in the equivalent section (i.e. the reinforced concrete deck, haunches, and steel girders). The mass per unit length is approximated based on typical values for density of steel 7800kg m3(:487kip ft3)and concrete 2400kg m3(:145kip ft3); all elements of the subject cross section, along with members not depicted (e.g. diaphragms and lateral reinforcement), are considered in this approximation. Damping is selected based on typical values for composite bridges outlined in the speciﬁcations [40]. Having obtained the MoI and area for the equivalent cross section of concrete, a square cross section is designed with the same equivalent properties to represent the system numerically, see Figure 2c for more details. The bridge is determined to be geographically located in Clemson, SC or Chicago, IL so that previously collected temperature data from these areas can be utilized to model temperature eﬀects, please see sub- section 3.2 for more details. The monitoring period for the bridge is 730 days, approximately two years. During this time, one or two vehicles (one traveling each direction) make a crossing every minute of each day; this was done to ensure adequate data is available for the NN process in Section 4. An inventory of eight separate vehicles is utilized for monitoring the health of the subject bridge; Table 2 indicates the properties for each vehicle. Vehicles are designed in such a way to represent, in a general sense, the dynamic properties of physical highway vehicles [37]. The resonant frequency for the upper and lower body of each vehicle is designed to fall between 1Hz2Hzand10H z15H z, respectively. The suspension damping is designed to have a ratio between 20%40%, while the wheel damping is designed to 7 Table 1: Bridge structural properties Length Mass per Length Elastic Modulus MoI Area Damping Ratio m (ft)kg m(kip ft) GPa (ksi) m4(ft4) m2(ft2) % 16 (52.5) 12000 (8.68) 26.40 (3829) 0.5152 (59.7) 5.12 (55.11) 2 Table 2: Vehicle properties Sprung Unsprung Suspension Wheel Suspension Wheel Mass Mass Stiﬀness Stiﬀness Damping Damping kg (kip) kg (kip)kN m(kip ft)kN m(kip ft)Ns m(kips ft)Ns m(kips ft) V1500 (1.10) 55 (.12) 37 (2.54) 350 (23.98) 2256 (.155) 25 (.0017) V2700 (1.54) 70 (.15) 60 (4.11) 350 (23.98) 3888 (.266) 40 (.0027) V3850 (1.87) 85 (.19) 80 (5.48) 600 (41.11) 5772 (.396) 60 (.0041) V41000 (2.21) 100 (.22) 115 (7.88) 400 (27.41) 8579 (.588) 90 (.0062) V52000 (4.41) 200 (.44) 300 (20.56) 1000 (68.52) 14679 (1.01) 150 (.0103) V64000 (8.82) 400 (.88) 900 (61.67) 2000 (137.04) 30000 (2.06) 300 (.0206) V78000 (17.64) 800 (1.76) 2500 (171.30) 4000 (274.09) 113137 (7.75) 1210 (.0829) V89000 (19.84) 900 (1.98) 2850 (195.29) 4000 (274.09) 128125 (8.78) 1300 (.0891) be approximately 1%of the suspension damping [37]. Variable traﬃc patterns are simulated by randomizing the sequence used to determine which one or two vehicles are currently traversing the bridge. The order and time history for each vehicle entering/exiting the bridge is randomized for the two vehicle case to prevent false relationships from being learned by the NN. To introduce variations in the measured bridge frequency and spectral resolution, vehicle speeds are randomized between 2m sand25m s(4:5mph56mph). Lastly, to account for changes in passengers and cargo, vehicle masses are allowed to vary by diﬀerent degrees. For small vehicles ( <1000kg), the mass is allowed to vary by 50kg (:11kip); for medium sized vehicles (<8000kg), the mass is allowed to vary by 100kg (:22kip); and for large vehicles ( 8000kg), the mass is allowed to vary by 200kg (:44kip). 3. Modeling Operational, Environmental, and Damage Eﬀects 3.1. Surface Roughness Proﬁle Surface roughness can alter the acceleration response of a vehicle in a manner that may mask all bridge frequencies. In this paper, the bridge’s surface roughness proﬁle is generated using power spectral density (PSD) functions deﬁned by ISO-8608:1995 standards. To use these standards, it is assumed the entire surface of the bridge deck falls under the same statistical classiﬁcation, meaning a single roughness class exists homogeneously across the entire deck [41]. There are eight classes used to describe surface roughness; the ﬁrst class (A) represents a smooth proﬁle with minimal roughness, while the last class (H) represents a poor proﬁle with high roughness [41]. In this project, the class A surface proﬁle is applied towards each bridge to ensure that the selected NN strategy is at least feasible for the most optimal surface conditions. Surface proﬁles are classiﬁed based on their PSD of vertical displacement ( GD), which is a function of spatial frequency ( ncycles m). Equation 24 deﬁnes GD(n)for a simulated surface proﬁle: GD(nj) =GD(n0)(ni=n0)2; (24) wherenjrepresents the jthspatial frequency being considered within a predeﬁned frequency band ( :01cycle m to10cycles m). The spatial frequency is sampled in intervals of1 L, whereLis the length of the bridge. GD(n0) represents the PSD of vertical displacement calculated as a function of traditional values of spatial frequency (n0=:1cycles m) [41, 42]. The ISO-8608 standard diﬀerentiates between surface classes based on their lower and upper bounds for GD(n0); for surface class A, the lower and upper bounds are 0m3and32e6m3, 8 (a) (b) Figure 3: An example demonstrating the variability in elevations that can occur when generating surface proﬁles for individual vehicle runs. (a): Power Spectral Density of class A surface proﬁles. (b): Elevations of class A surface proﬁles. respectively. In this study, the value for GD(n0)is selected to be 4e6m3. The surface amplitudes are calculated in Eq. 25, and then entered into Eq. 26 to determine elevations at locations along the bridge: Aj=2q 2GD(nj)4n (25) h(x) =NX j=0(Aicos(njx+ j)); (26) where jis a random phase angle following a uniform probabilistic distribution between the values 0and 2. Parameter xis the global position within the range 0toL, and4nrepresents the sampling interval of the spatial frequency ( 1=L) [41, 42]. Vehicle velocity and modeling time-step govern the number of elevation points in a surface proﬁle. For a ﬁxed time step, an increase in velocity decreases the number of elevation points, and a decrease in velocity increases the number of elevation points. In this paper, velocity is varied between 2m sand 25m s(4:5mph56mph), making it diﬃcult to simulate a homogeneous proﬁle that has consistent global elevations across test runs. What’s more, the randomly assigned phase angle jfurther increases variability in elevations across simulations. Figure 3 demonstrates this issue for two sample surface proﬁles. Figure 3a shows the Power Spectral Density (PSD) of the proﬁles and demonstrates that despite having diﬀerent amplitudes for spatial frequencies, they both follow a class A classiﬁcation. Figure 3b displays the change in elevation over bridge length, and demonstrates that the two proﬁles diﬀer signiﬁcantly from one another despite being generated with the same vehicle speed and time step. To prevent surface proﬁles from continuously changing across simulations, a single surface proﬁle is generated that can be applied towards all test runs regardless of vehicle speed. This is done by employing a velocity of 1m s(2:24mph)and a modeling time step of :001sto obtain surface elevations for every thousandth of a meter along the subject bridge’s surface. By having such a ﬁne surface mesh, the precise elevation corresponding to any vehicle’s global coordinate on a bridge can be extracted for each time step. If Proﬁle 1 from Figure 3 is re-sampled at a velocity of 10m s(22:4mph), it is observed, in Figure 4, that the spatial frequency content and the elevation magnitudes now remain constant. Surface roughness eﬀects are integrated into the ﬁnite element model by altering the equations for wheel displacement, velocity, and acceleration (Eq. 19-21), and by altering the ﬁnalized equation of motion for the VBI element (Eq. 23). Equations 27-30 show these new calculations: dw=fNgTfdbigt+4t+h(x) (27) _dw=f_NgTfdbigt+4t+fNgTf_dbigt+4t+vh0(x) (28) dw=fNgTfdbigt+4t+f_NgTf_dbigt+4t+fNgTfdbigt+4t+v2h0(x) (29) 9 (a) (b) Figure 4: An example demonstrating how surface proﬁle elevations remain constant despite changes in vehicle speed. (a): Power Spectral Density of class A surface proﬁles. (b): Elevations of class A surface proﬁles.  [mbi]fdbig+ [cbi]f_dbig+ [kbi]fdbig t+4t=  ffbig[M c]fdbig[C c]f_dbig[K c]fdbigfP cg+fH cg t+4tfQ cgt(30) wherevis the velocity of the vehicle(s), and h0(x)is the derivative of the surface amplitude with respect to x. VectorfH cgrepresents the eﬀect of surface roughness in the contact force between the vehicle(s) and bridge, and is calculated using Eq. 31 below: fH cgt+4t=fNg v2Mch00(x) +vCch0(x) +Kch(x) : (31) 3.2. Temperature Eﬀects In this section, focus is placed on modeling temperature induced variations in DBHM data. Thermody- namics indicate that a linear correlation exists between temperature and bridge elastic modulus and natural frequency [43], a trend that researchers have found holds true for many reinforced concrete decks [44, 45]. However, bi-linear relationships have also been observed for bridges where an asphalt wearing surface causes diﬀerent rates of softening and stiﬀening, such as the case of the Z-24 bridge in Switzerland, or where interior moisture freezing causes deck stiﬀness to increase at a linear rate diﬀerent than softening, such as the case of the Dowling Hall Footbridge at Tuﬀs University [26, 27]. Thus the relationship between temperature and bridge modulus or frequency can be linear or nonlinear in nature, and is dependent upon a bridge’s environmental and operational conditions. Because the constitutive equations required to simulate the governing relationship between temperature and a bridge’s dynamic characteristics are mostly unknown, and the number of parameters that must be considered is vast, simulating thermodynamic eﬀects is a diﬃcult task [46, 47]. To get around this, models based on measurements from real structures can be leveraged to portray the relationship between temperature and variations in a bridge’s dynamic response [47]. In this paper, we employ bi-linear equations for bridge elastic modulus developed for the Dowling Hall Footbridge [48]. Equation 32 calculates the modiﬁcation factor for a subject bridge’s elastic modulus, which is then multiplied to the bridge modulus to obtain the temperature shifted modulus value [48]:  =Q+STt+R 1erfTt  (32) Em= E0; (33) whereQandSTtare hyperparameters that represent the linear behavior of the bridge above freezing, while R(1erf(Tt ))represents the approximate linear behavior at or below freezing. Because the behavior between frequency and temperature has more variability at lower temperatures, the Matlab error function 10 Table 3: Mean and standard deviation for temperature variables QSR Mean 1:01290:0048 0:19771:1012 3:1466 STDV 0:003 0:0001 0:0027 0:0513 0:0861 (erf) is employed to introduce higher variance at these temperature settings. Ttis the temperature in degrees Celsius at time t, while variables and are hyperparameters that represent the area for transitioning from below freezing to above freezing. E0is the nominal bridge elastic modulus, while Emis the modiﬁed elastic modulus. Table 3 shows the mean and standard deviation values for each of the hyperparameters above [48]. Although these equations were developed for a speciﬁc bridge, they are deemed appropriate for this study as the deck of the Dowling Hall Footbridge is constructed using a traditional reinforced concrete design with no wearing surface, analogous to most highway bridges in the United States. In this study, hourly temperature data, provided by the National Centers for Environmental Information forthecitiesofClemson, SCandChicago, IL,isusedtocreaterealisticdailyandyearlyoscillationpatternsin bridge elastic modulus and frequency [49]. In the Clemson data set, there are little to no hourly temperature values that fall below zero; meaning, the Clemson data set represents a scenario where only above freezing temperatures are present year round (i.e. only a linear relationship exists between bridge frequency and temperature). In the Chicago data set, temperatures ﬂuctuate between freezing and above freezing year round (i.e. the bi-linear relationship between bridge frequency and temperature exists). By employing the two diﬀerent sets of temperature data, we are able to test if the accuracy of the health monitoring approach is aﬀected by the complexity of the temperature-frequency relationship. Linear interpolation provides temperature values for simulations occurring between the top and bottom of each hour. Because the bridge is short-spanned, it is assumed to have consistent exposure conditions and, therefore, uniform temperatures throughout. It is also assumed that temperature does not aﬀect the dynamic properties of vehicles, and that damage does not cause a change in the bi-linear relationship between temperature and bridge modulus. 3.3. Modeling Damage In this study, damage is introduced in the form of various levels of cracking. Research has shown that the impact of crack damage on the resonant frequencies of a bridge can range from insigniﬁcant to very noticeable, andisdependentonthelocationofdamage, lengthoftheimpactedarea, andcrackdepth[50,51]. A variety of studies have attempted to determine if the magnitude of change in the principle frequency can be used to identify the severity of damage within a structure, Table 4 demonstrates the ﬁndings from studies conducted on FE models and lab systems. In a study conducted on a full scale three-span simply supported bridge, Salane et al. found that the principle frequency decreased by 7:3%,8%, and 16:7%for respective fatigue test cycles of 9:5e+4,2:15e+5, and 3:735e+5. In the aforementioned study, information regarding the Table 4: Studies demonstrating the percent change in bridge like structure frequencies caused by crack damage Crack Percent Reduction Percent Impact Percent Reduction Reference Depth in MoI Length in Frequency mm (in) % % % 28 2:5 :7 [52] N.A. 28 10 2:85 3:2 (:125) 19 :95 [53] 25:4 (1) 32 N.A. 4:2 49:5 (1:95) 33 10 11:2 (:44) 40 1 4:1 [54] 11:2 (:44) 40 17 12 5 (:196)-Half Flange Width 10 1:2 [55] 5 (:196)-Full Flange 50 N.A. 5:6 11 Table 5: Depth of cracking in relation to percent change in moment of inertia and principle frequency Crack Percent Girder Reduced Percent Reduction Percent Frequency Depth Depth MoI in MoI Change mm (in) % m4(ft4) % % D111.2 (.44) 1.65 .4652 (49.54) 9.70 .50 D219.63 (.77) 2.90 .4276 (42.20) 17.00 1.00 D328.24 (1.11) 4.17 .3890 (36.59) 24.50 1.50 D433.80 (1.33) 4.99 .3642 (32.20) 29.30 2.00 D540.88 (1.61) 6.03 .3328 (28.71) 35.40 2.50 severity of cracking was not provided; however, it was stated that the stiﬀness of the structure changed on average by 20%and by a max of 24%, with the max reduction being associated with a completely fractured girder [56, 57]. Other studies conducted on full scale bridge structures have demonstrated crack damage causing noticeable changes in the principle frequency of bridges; these studies, however, also demonstrate the eﬀect temperature can have on masking changes caused by damage, therefore making it diﬃcult to associate a level of cracking with a percent change in frequency [21, 26, 58, 59]. As indicated by the studies above, the relationship between severity of crack damage and changes in principle frequency vary from structure to structure, implying that the relationship is dependent on the boundary conditions, geometry, and material properties of a structure. This appears to hold true as, to our knowledge, no generally accepted metric has been established for identifying the relationship between levels of crack damage and changes in the principle frequencies of highway bridges. Because of this, we assume a relationship where an approximately 50%reduction in moment of inertia, applied on the central 5%of the bridge’s length, results in a 5%change in principle frequency; this relationship is in the family of the magnitude of frequency change and MoI reduction observed from the studies listed in Table 4. For this relationship to hold true, the equivalent beam section has to be discretized into 21 ﬁnite elements (i.e. each element is:76m (2:5ft)). Note that damage is not introduced in the form of a 50%reduction in moment of inertia, this level of damage is just to establish the relationship between damage and reduction in frequency. In this study, damage is represented using two diﬀerent cases. In the ﬁrst case, damage is introduced as ﬁve instantaneous decreases in moment of inertia on the central element of the structure. Each level of damage corresponds to a 0:5%change in natural frequency. The initial level of damage represents a situation where minor localized damage has occurred, while the ﬁnal level represents a situation where the bridge has sustained severe localized damage and safety has become an issue. Damage is considered localized as it only aﬀects the central element of the structure, correlating to an impact length of 4:75%of the total length. The initial decrease in moment capacity corresponds to tension cracks forming in the lower ﬂange of the four interior girders, while each decrease thereafter corresponds to the tension cracks continuing to propagate through the ﬂanges. Table 5 shows the depth of cracking required on each of the four interior girders in order to cause each respective decrease in moment capacity; Figure 5 provides a visual depiction of this process. Each level of damage respectively occurs at 25%,33%,50%,66%, and 75%of the way through the approximately two year monitoring period. These ranges of time were selected so that an adequate amount of data could be collected for each level of damage. It should be noted that temperature eﬀects are still considered for the damaged bridge element. Figure 5: Levels of damage in cross section of four interior girders. 12 Table 6: Length of damage in relation to percent change in principle frequency Crack Reduced Length of Percent of Percent Frequency Depth MoI Damage Total Length Change mm (in) m4(ft4) m (ft) % % D119.63 (.77) .4276 (42.20) .76 (2.5) 4.75 1.00 D219.63 (.77) .4276 (42.20) 1.52 (5) 9.5 1.96 D319.63 (.77) .4276 (42.20) 2.28 (7.5) 14.5 2.88 D419.63 (.77) .4276 (42.20) 3.04 (10) 19 3.74 D519.63 (.77) .4276 (42.20) 3.8 (12.5) 23.75 4.53 For the second damage case, cracking no longer occurs locally on the central element, but rather expands outward from the central node along the length of the structure. Damage is still introduced as ﬁve instan- taneous decreased in MoI, but now each instance of damage is associated with a 4:75%increase in impact length instead of an increase in crack depth. To allow damage to expand evenly outward from the central node, while only causing a roughly 5%increase in impact length, the structure is discretized into 40ele- ments; meaning, the length of each element is now :38m (1:25ft)and accounts for 2:375%of the total length. The initial level of damage occurs on the central two elements in the form of damage level two outlined in Table 5; thereafter, the same level of cracking spreads to the adjacent two elements to cause the percent length of impact to increase. The ﬁnal level of damage represents a situation where approximately a quarter of the bridge’s length has been aﬀected by cracking. Table 6 shows the length of the structure impacted for each level of damage; Figure 6 provides a visual depiction of this process. Each level of damage respectively occurs at 25%,33%,50%,66%, and 75%of the way through the approximately two year monitoring period. Again, temperature eﬀects are still considered for the damaged bridge elements. 4. Machine Learning Modeling and Implementation Details 4.1. Machine Learning Problem Formulation Despite being one of the most popular indicators for bridge damage, changes in natural frequencies caused by damage tend to be small and are often masked by operational and environmental noise [14]. These limitations suggest that NNs, which can provide fast inferences as well as noise-tolerant approximate results, could be an eﬀective solution strategy. In many other application areas, neural networks have been shown to eﬀectively approximate complex simulations [60, 61]. In these cases, a computationally expensive routine has been replaced by a reasonably high-quality approximation, which reduces the overall time-to- solution by multiple orders of magnitude. Similar networks are also noise-resilient across a number of tasks, such as computer vision [62]. Convolutional models, those that simplify their input by pooling iterative convolutional ﬁlters, are a key component of many such computer vision tasks [63]. Figure 6: Levels of damage along percent length of subject bridge. 13 One such model, VGG19 [64], is able to eﬀectively classify images in the ImageNet dataset by applying multiple “block” of two convolutional layers followed by one pooling layer. The model presented here, and depicted in Figure 8 follows the same pattern as VGG19, but reformulates the model architecture to better understand the vibration data gathered from vehicles crossing bridges. The main diﬀerence arises from our choice of one-dimensional convolutional ﬁlters, as they better map to the frequency-amplitude domain. However, similar to VGG19, we apply blocks of two layers of convolutional ﬁlters, with a ﬁlter width of 3, followed by a max-pooling layer that reduces the eﬀective size of the data by half each operation. Subsequent blocks double the number of convolutional ﬁlters, progressively moving from a “tall” input, to a “wide” intermediate representation. This representation is then “ﬂattened” so that we may apply two fully connected layers that are capable of learning a non-linear relationship between the intermediate representation and the six damage classes. The process of applying ﬁlters and pooling results introduces signiﬁcant redundancy into the neural architecture, which improves the overall resiliency of the NN model, and leads to high performance results. 4.2. Feature Engineering In machine learning, a “feature” refers to an observable characteristic of something to be modeled [65]. Ultimately, a machine learning model must learn the relationship between these observable characteristics and some resulting characteristic of interest. In this study, features must be selected that foster the estima- tion of bridge health; however, the drive-by monitoring paradigm limits the number of features to just those that can be observed via sensors placed on a moving vehicle. It is assumed a monitoring vehicle will have accelerometers on both the sprung and unsprung masses that are capable of capturing vertical acceleration, a thermometer to capture ambient temperature, and a speedometer to measure average speed. Additionally, because the exact mass of each vehicle is unknown due to variable cargo and passengers, vehicle masses are approximated according to the values provided in Table 2. This is equivalent to the real-world scenario where vehicle mass could be approximated as the value provided by a vehicle’s manufacturer. In this study, vertical acceleration data from both the upper and lower masses of a passing quarter-vehicle model are captured over time. The sequence of vertical acceleration data obtained from the vehicles is pre- processed by ﬁrst applying a sharp low-pass ﬁlter at 100H zto prevent aliasing and reduce high frequency noise eﬀects. An FFT with a band-pass ﬁlter is then applied to identify the frequency spectrum of the crossing vehicle(s) within the range of 3Hz10H z; the length of each vehicle’s transformed signal is modiﬁed such that the spectral resolution equals 0:1Hz. This process results in 140 observations of frequency and amplitude for both upper and lower masses. Appended to each observation is the time of day the crossing occurred (out of 24 hours) and the vehicle’s mass provided in Table 2; again, the appended mass and the actual mass used in simulation are diﬀerent. These six features, frequency (shared by both masses), upper amplitude, lower amplitude, vehicle mass estimate, vehicle speed, and time of day, form the base of each observation. However, in the simulation scenarios wherein environmental eﬀects are introduced, a seventh feature is also added: the observed ambient temperature. These six or seven features are then placed in a sequence for each recorded vehicle crossing. It is these sequences that form the inputs of the NN model. It is important to note that the NN model has no direct notion of time-of-year, and that during training we randomize the order of observed vehicles. The ability to learn an indirect relationship between temperature and time-of-year is obfuscated by running the simulation for approximately two years. This is an important detail as any trivial model could learn a simple relationship between damage and time. In contrast, we ensure the model must learn to detect signals present in vehicle frequency spectra. The previously mentioned band-pass ﬁlter is employed to prevent dominate vehicle frequencies from aﬀecting the health monitoring results. As can be seen in Figure 7a, the amplitudes associated with vehicle modesdominatethefrequencyspectrum, potentiallyaﬀectingthelearnedrelationshipswithintheNNmodel. To prevent any issues with vehicle frequencies aﬀecting the NN, the range for the band-pass ﬁlter is selected such that all modes associated with vehicles are excluded from the frequency spectrum. The frequency range for the ﬁlter is not solely selected based on vehicle frequencies, but rather is also selected considering initial estimates for bridge frequency and variations in frequency caused by operational and environmental eﬀects. Because we have prior knowledge of the subject bridge’s cross section and material properties, we are able to determine its principle frequency. However, an empirical formula has been developed that allows 14 (a) (b) Figure 7: Example of how vehicle frequencies dominate frequency spectrum, and how band-pass ﬁltering can be employed to remove vehicle frequencies from response spectrum. (a): Raw frequency spectrum for sprung body (top) and unsprung axle (bottom) masses (i.e. no band-pass ﬁltering has been applied) (b): Filtered frequency spectrum for sprung body (top) and unsprung axle (bottom) masses (i.e. band-pass ﬁltered has been applied). the principle frequency of a simply supported bridge to be approximated when it is unknown, see Eq. 34 below [66]: f1= 100=L: (34) For this study, the estimated principle frequency falls around 6:25H z, which is close to the FE model value of 6:53H z. In their study, Haywood et al. also provide upper and lower bounds to demonstrate the approximate range the principle bridge frequency may fall within for a select span length; for the span length provided in Table 1, the lower and upper bounds are approximately 5Hzand8Hz, respectively [66]. This range is expanded to 3Hz10H zto account for large variations in oberserved principle frequency caused by temperature eﬀects and vehicle speed. As can be seen in Figure 7b, the selected width for the band-pass ﬁlter excludes sprung and unsprung vehicle frequencies, and provides a suﬃcient enough range for capturing the subject bridge’s principle frequency. Figure 7b also demonstrates, however, that noise is still present within the captured frequency band, making it diﬃcult to discern which peak is related to the bridge frequency. Because of this, the amplitude value of all frequencies within the 3Hz10H zband band are collected and observed. Additionally, there are a number of potential features omitted in order to ensure a model that can generalize to diﬀerent vehicle and bridge conﬁgurations. These features can be grouped into two classes, those that are observable but not helpful, and those that are helpful but not observable. The former group consists of features such as the time of day or year a vehicle is present on a bridge. If time were included, the NN model would most likely learn a linear inverse correlation between time and health, and in doing so would discard all other information. Clearly, this would not be a preferable model. The latter group of features, those that cannot be observed, include bridge properties such as mass and surface elevations. While these features would certainly improve the model, we do not assume they are detectable from a vehicle outﬁtted with the aforementioned health monitoring sensors. Figure 8 depicts the neural architecture used to determine bridge health for each simulated vehicle. As previously mentioned, the architecture follows the paradigm established by VGG19. In this ﬁgure, the red bands represent a section of values read by a single application of a convolutional ﬁlter. The blue dots indicate that a single value in the downstream layers is derived from the entire set of values highlighted in red. The yellow layers represent the max-pooling operation, where adjacent values are compared and the higher is selected to participate in the following layers. The ﬁnal yellow block is “ﬂattened”, meaning it is reshaped to be accepted by the following fully connected layers. The size of each layer is deterministic based on the size of the input. Each convolutional layer reduces the “height” of the intermediate sequence by two (because of convolutional ﬁlters of size three). Each pooling layer reduces the height by half, because pooling compares disjoint sets of adjacent features. By convention, following each pooling operation, the number of convolutional ﬁlters is doubled (in this case, the “width” of 15 Figure 8: The above depicts the neural architecture that predicts bridge damage classes given features corresponding to a single vehicle’s crossing. Information ﬂows left to right, starting with the sequence containing FFT observations for both upper and lower masses in the quarter vehicle model. The size of this input layer is derived from the sampling rate and band-pass ﬁlter width of our FFT. Each white box then indicates a 1-dimensional convolutional ﬁlter, while each yellow box corresponds to a max-pooling operation. Towards the end, we ﬂatten our convolutional layer and apply two fully connected layers of size 512, before ﬁnally predicting one of six damage classes. a layer). The only exception to this is the ﬁrst layer, which expands the input from six or seven features to 16. The size of the two fully connected layers is determined experimentally. While VGG19 uses two layers of size 4096, it is prone to overﬁtting. Using smaller fully connected layers drastically reduces this problem by limiting the solution space of the overall model to simpler combinations of intermediate values. Addition- ally, because the problem uses fewer input features (140x7 vs 244x244x3), simpler underlying relationships between input to output are expected. The NN model uses ReLU as the activation function for every layer, except the ﬁnal layer which applies softmax. The model is ﬁt using the Adam optimizer, a batch size of 128, and has an early stopping criteria of ten epochs to prevent overﬁtting (model training stops if the validation performance does not improve for ten iterations of the training set). The weights of the NN model are updated through back-propagation of the categorical cross-entropy loss function. The resulting mode that is used for evaluation is whichever version best minimizes the loss function on the validation set following an iteration of the training set [29]. 4.3. Evaluation Strategy Themodelsareevaluatedbytheabilityoftheproposedmachinelearningsolutiontodetectbridgedamage levels using a 20%holdout experiment across an array of simulation scenarios. Each scenario represents a diﬀerent conﬁguration of simulation parameters. The parameters considered are: the number of vehicles (1 or 2) simultaneously present on the bridge, the damage pattern (expanding or localized), whether or not environmental eﬀects are considered (on or oﬀ), and two diﬀerent weather models (real data from either Clemson SC or Chicago IL) corresponding to diﬀerent temperate zones in the continental United States. Overall this evaluation spans all 16 scenarios. First two years of bridge crossings are simulated at diﬀerent times of day across all scenarios. Data from reach individual crossing is randomly shuﬄed and divided into training, validation, and testing sets by a 70%10%20%split. Test-set performance is calculated in two ways, both per-vehicle and per-day. The per-vehicle evaluation metric is accuracy, which determines whether an arbitrary vehicle can predict bridge health provided a single crossing. The per-day evaluation metric is “ conﬁdence ”, which aggregates the total crossings for a single day and determines whether the model’s damage predictions are correct across a majority of trials. In this case, the prediction of the model for a single day is equal to the damage class receiving a majority vote from that day’s simulated crossings. Conﬁdence evaluates how signiﬁcant is the majority vote of damage class from a single day, and whether that vote results in an accurate bridge health estimate. This metric ranges from one, where all vehicles predict the correct class, to negative one, where all vehicles predict the same incorrect class. A conﬁdence of zero indicates that the vote was tied. However, in the results presented here all conﬁdence values are positive as the NN solution always produced correct per-day predictions. Formally, this metric is deﬁned as: Cz=P(Tz;z)max d2DTzP(d;z): (35) 16 WhereTzis the true damage class of the considered bridge on day z,P(d;z)be the proportion of vehicles on dayzthat predict damage class d, andDis the set of damage classes. Put plainly, Czis equal to one if every vehicle predicts the damage class accurately. Czis equal to zero if the number of vehicles predicting the correct damage class equal the number of vehicles predicting some incorrect class. Czis only less than zero if some other class is preferred to the true damage class. In our later analysis, we plot daily conﬁdence over the course of our two simulated years to show the model’s ability to detect bridge damage in diﬀerent conditions. 5. Results For each of the 16 simulation scenarios described above we generate approximately two years (730 days) worth of bridge crossings, split the set of crossings into training, validation, and testing sets, train a convolutional NN model, and evaluate the test set in terms of per-vehicle accuracy and per-day conﬁdence. These results are presented across all scenarios in Table 7. Results are compared using accuracy to judge the overall performance of a NN model. Many of the results achieve an accuracy of one, indicating that every vehicle in the test set is able to correctly identify bridge damage. As accuracy decreases, so too does conﬁdence. The provided conﬁdence plots indicate the sorts of bridge damage that a model is better or worse at detecting. The conﬁdence plot is a dense bar plot, depicting conﬁdence from zero to one on the vertical axis, and the day from 1 to 730 across the horizontal. The color of each region indicates the bridge damage class, from healthy (green) to maximum damage (red). If the height of the colored region decreases, that indicates that more vehicles incorrectly detected the current damage class for some common incorrect class. Of the parameters varied across simulation scenarios, the number of vehicles simultaneously present on the bridge is the most impactful. All models trained on data where only one vehicle is present per crossing achieve an accuracy of over 0.99, while none of the scenarios with simultaneous crossings reach that same accuracy. The decrease in accuracy as a result of multiple simultaneous vehicles is expected. Data is only recorded from a single vehicle per-crossing, and so there is no way to detect the unobserved vehicle in the NN model. In addition, the unobserved vehicle’s crossing has no relationship to the observed vehicle — it may cross before or after, or not at all. Therefore, the performance gap between the single- and double-crossing scenarios is indicative of the performance diﬀerence made by noisy training data. The results demonstrate that the average decrease in accuracy is 0.111 when comparing a one vehicle trail to its reciprocal two vehicle counterpart. When comparing conﬁdence plots, it is observed that this decrease in performance is least seen when the bridge is healthy, and in many trials is most prevalent for the intermediate damage classes. When comparing the simulation scenarios across the damage pattern parameter, it is observed that the expanding damage pattern is slightly easier for the NN model to capture. While this eﬀect is less pronounced than the number of simultaneous vehicles, Table 7 indicates that the model accuracy is 0.0693 less when comparing a scenario with localized damage to its corresponding expanding simulation. However, this diﬀerence primarily almost entirely derives from the noisy two vehicle case, where diﬀerences in performance are more pronounced (average absolute diﬀerence among the single-vehicle trials is only 0.00275). Again, the largest diﬀerence in quality is observed to occur in the intermediate damage classes. When evaluating the eﬀect that environmental factors such as temperate have on the NN model, there is diminished impact again. Now, the average decrease in accuracy is only 0.0115 when comparing a simulation with no environmental eﬀects to one with environmental eﬀects enabled. Because the NN model receives temperature as an input per-crossing, it is able to better compensate for its eﬀect, as compared to the other simulation parameters. Additionally, in contrast to parameters such as the number of simultaneous vehicles, the recorded temperature has a more predictable, and therefore discoverable, relationship. However, the NN does appear to have overﬁt on temperature in some areas, such as the “dip” in the ﬁnal damage class or at the end of the healthy class, which are only observed when environmental eﬀects are enabled. This eﬀect is due to an incorrectly learned relationship between temperate and health, although its eﬀect is minimal. Lastly it does not appear that the location of weather data has much impact on simulation results. When comparing trials with environmental eﬀects enabled, Clemson weather data leads to an average of 0.00125 17 Table 7: Accuracy and conﬁdence for each simulation scenario Num. VehiclesDamage PatternEnv. Weather LocationAccuracy Conﬁdence Plot 1 Expanding Oﬀ Chicago 1.000 1 Expanding Oﬀ Clemson 1.000 1 Expanding On Chicago 0.997 1 Expanding On Clemson 0.992 1 Localized Oﬀ Chicago 1.000 1 Localized Oﬀ Clemson 1.000 1 Localized On Chicago 1.000 1 Localized On Clemson 0.991 2 Expanding Oﬀ Chicago 0.963 2 Expanding Oﬀ Clemson 0.960 2 Expanding On Chicago 0.950 2 Expanding On Clemson 0.950 2 Localized Oﬀ Chicago 0.828 2 Localized Oﬀ Clemson 0.830 2 Localized On Chicago 0.795 2 Localized On Clemson 0.814 The column of conﬁdence plots graphically describes model performance as a dense bar chart over the two-years of simulation time. The vertical dimension represents conﬁdence between 0 and 1 and the horizontal dimension represents time. Additionally, the color of the segment indicates damage class, where green indicates the undamaged bridge, and progressing colors indicate damage levels, ending at the highest damage case in red. higher accuracy, which is negligible. Interestingly, many of the same weather-related overﬁtting behavior across similar date ranges across related trials with diﬀerent underlying weather data. This is likely due to the normalization applied to all of the training data, which standardizes all input features to the zero to one interval. Therefore, the two cities yearly weather patterns indicate similar signals, despite existing in very diﬀerent ranges. Across all trials, the NN model is able to capture relevant relationships between the vehicle’s measure- ments and the bridge’s health. While there are mild variances in model accuracy for a single vehicle, the fact that every day exhibits a positive conﬁdence value indicates that when a simple majority vote pooling method is applied, the NN model is always able to correctly predict bridge health in the simulation. 6. Conclusion The purpose of this study was to demonstrate the feasibility of a new DBHM approach (i.e. training neural networks on vehicle-bridge simulations to detect damage on physical bridge systems) by determining if an actionable health assessment could ﬁrst be obtained from generalized yet realistic numerical models of vehicle-bridge systems exposed to environmental and operational noise. A ﬁnite element methodology based on the concept of vehicle-bridge interactions was used to simulate a simply supported bridge subjected to dynamic loads introduced by moving quarter-vehicle models. Environmental and operational noise were represented by vehicle traﬃc, surface roughness, and temperature induced changes in bridge elastic modulus. 18 Damage was introduced as instantaneous decreases in moment of inertia for either a single bridge element (localized) or multiple elements (expanded). As damage is known to cause variations in bridge frequencies, the frequency content contained within the acceleration response of each vehicle was extracted. Because operational and environmental eﬀects can introduce high amplitude noise or shifts in the frequency response spectrum, itwasdiﬃculttodistinguishbetweenwhatwasabridge’smodalfrequencyornoise. Toaccountfor this, amplitudevaluesforfrequencieswithinthe 3Hz10H zrangewereextractedfromthefrequencyresponse data. Other data, such as ambient temperature, speed, and approximate vehicle mass, was appended to the extracted frequency data. A multi-layer convolution neural network was then leveraged to analyze the aforementioned data and provide a noise-tolerant relationship between observations and bridge health. To test the NN’s ability to generalize and provide insight into the methodologies performance, a multi-class system study was conducted that assessed the structural condition of the subject bridge based on its level of theorized damage. The multi-class study was conducted for a series of 16 diﬀerent simulation scenarios, with each scenario representing a diﬀerent conﬁguration of simulation parameters. The results from the multi-class study demonstrate the ability of the convolutional NN to detect bridge damage across a range of real-world noise signals. When comparing idyllic simulation scenarios, wherein NN modeling can perfectly uncover bridge damage, to similar scenarios that incorporate real-world noise signals, thisstudyﬁndsmarginalperformancediﬀerences. Whileotherunmeasuredvehiclesposethegreatest challenge, it and other complicating factors, such as diﬀerent climates, temperatures, and damage patters, can be automatically discarded by a convolutional model. Although this study cannot directly evaluate whether a similar approach could counteract every source of real-world noise, its results do highlight the noise-canceling ability of machine learning models across a number of important factors. Furthermore, simple temporal pooling, such a daily voting mechanism, allows for the proposed machine learning approach to overcome all considered noise sources and discover the true bridge health state. 7. Future Work The results from this study indicate that the proposed DBHM approach is feasible and should be further explored as a viable health monitoring strategy. Before being able to apply this approach towards physical structures, however, studies need to be conducted to demonstrate the ability of simpliﬁed vehicle-bridge models to reliably capture the dynamic response of physical structures. More importantly, focus should be placed on identifying and validating methods for modeling the various types of damage that can occur in bridge structures (e.g. breathing cracks and changes in boundary conditions). Finally, studies should be conducted to ascertain the maximum level of surface roughness and the minimum level of daily traﬃc that will allow for accurate damage classiﬁcation. Acknowledgements The authors gratefully acknowledge the support of the National Science Foundation under grant #1633608.