This work describes a combination of image analysis techniques used to identify vehicles travelling on a bridge with a vectorised modal dynamic analysis that can handle efficiently a large number of wheel loads on the deck at each analysis step-time. In the absence of weight-in-motion data, a randomisation of the traffic flow is proposed to define the weight of the vehicles as a function of their identified size. The methodology is applied to real CCTV recording on a conventional road bridge with a large width- to-span ratio in which the deck is modelled with shell elements. The latter is found to be important to capture the significant contribution of local slab modestothevibrationsalongthesidewalks. Thedynamicanalysisofalarge numberoftrafficrecordsindicatethatcode-basedloadcaseswithlongtruck convoys lead to unrealistically large contributions of high-order modes and tovibrationsthatarecategorisedasuncomfortable,whilstthemorerealistic traffic flows obtained from image analysis satisfy the comfort criteria based on root-mean-square accelerations. keywords: Modal dynamics, traffic flow, bridge vibrations, comfort assess- ment, computer vision. 1 1 Introduction Over the service life of a road bridge, real live actions consist of millions of fluctuating loads caused by the objects, mainly vehicles, that traverse over it. The vehicles can modify their speeds, change lanes or stop completely, depending on the actual traffic conditions, and it is essential to represent these accurately in fatigue life assessment, vehicle driving safety and users’ comfort analyses. In fatigue analysis, the traffic actions are traditionally over-simplified as sce- narios of vehicles or convoys that are static or move at constant speeds [1, 2, 3], which hardly resemble the real vehicle actions. As a result, there is consensus on theneedtoimprovethedefinitionoftheloadinginfatiguelifeassessments[2,4]. On the other hand, most of the previous works on the driving safety and comfort analysis of road bridges consider a single moving vehicle at constant speeds (e.g. [5,6,7,8,9,10,11,12,13,14,15,16,17])oraconvoyofequallyspacedvehicles movinginstraightpaths(e.g. [18, 19,20,21,22]). Consequently,thereisagrow- ing interest on the definition of realistic traffic actions in bridges that account for theinteractionsbetweenvehiclesonthebridge. Tothisend,someresearchworks microsimulate the movement of individual vehicles within traffic flows generated with cellular automata (CA) algorithms that discretise the space of the road into cells that can be either occupied by a vehicle or empty [23, 24,25,26,27,28]. However, the discrete space-time nature of the CA vehicle models can lead to vehicle flows that are unrealistic for the dynamic analysis of the bridge. Veryrecently,theuseofCCTVvideorecordingshavebeenproposedtoobtain information about the actual traffic flows to which bridges are subject. Lee and Koh[29]usedreal-timevideotoidentifythepositionofthevehiclesandcorrelated itwithsensorsthatmeasuretheresponseofthebridge. Chen et al.[30]proposed animageprocessingtechniquetoobtainfromCCTVmonitoringthepositionand velocity of vehicles crossing bridges. Unfortunately, this information was not appliedtothestructuralanalysisofabridge. Itcanbeattributedtothesignificant computational cost involved in the dynamic analysis of structures subject to long and stochastic traffic flows with large numbers of moving vehicles, which are needed to obtain representative results of the structural response [26, 27,28]. The present research contributes a methodology that enables the CCTV mon- itoring system existing in most bridges to identify the position, speed and size of thousands of vehicles in real time, and feeds this information to a detailed structural analysis based on shell-like discretisations of the deck. In the absence of weigh-in-motion data, a randomisation of the identified vehicles’ weight based on their size is proposed. In addition, efficient load-interpolation, vectorisation 2 and sub-structuring techniques are also presented to reduce the computational cost of the modal dynamic analysis, and to enable the study of the response of bridges subject to long recorded traffic flows. This methodology is applied to a conventionalroadbridge,anditisobservedthatcomparedwiththerecordedtraffic flowsthetraditionalloadcasesbasedonequallyspacedconvoysofidenticaltrucks lead to unrealistic vibrations along the sidewalks, both in terms of magnitude and frequency content. 2 Methodology and case study The methodology proposed in this study combines image analysis with structural dynamics to obtain the response of bridges in time domain. Fig. 1 presents the analysis flowchart, which will be described as it is applied to the study of a road bridge monitored through a webcam. We select the case of the Old Evripos Bridgein Greece because it is constantly monitored (http://olne.gr/el/evripos- bridge/evripos-bridge-live-stream), with the camera and the website where the videos are accessible being operated by the Evia Island Ports Authority (OLNE S.A. https://olne.gr/). Fig. 2(a) shows the structure from a frame of a video taken by the webcam. It is noted that the real truss structure of the Old Evripos Bridge is not considered in the study. Instead, a more common ladder-deck composite bridge of the same span and width is proposed in order to increase the generality of the results obtained from the dynamic analysis. 2.1 Image analysis First, the frames of the video recording of the bridge and its traffic are processed. These frames are used to detect individual objects (cars, vans, trucks) that move along the bridge in two directions. The pre-processing of the video consists of a pipeline of image processing techniques [31, 32,33] with the following steps: 1.Frame separation: Frames of the video are selected as separate images. Due to memory constraints, for short videos ( 𝑡𝑎<60s) all frames can be considered. For longer videos, four frames per second are suggested. 2.Pixelstatistics: Mean,medianandstandarddeviationalongtimearecalcu- latedforeachpixeloftheframe,andusedtocreateamedianimage(Fig.2(b)) and standard deviation image (Fig. 2(c)). The median image gives a rep- resentative frame where all the moving objects have been removed. This 3 Set first time-step: ti=ti-1+t Initialise load vector: P=0Consider first wheel on the deck: l Obtain wheelposition: X w, l Search for adjacent shell nodes and obtain nodal loads:< ?tita YesNo =1< ? Yes NokNr < ?lNwi=1 = +1l lNext wheel:t0=0s Time loop Wheel loop P = P + PwYw, l, Update load vector:YesNo = +1k kNext traffic record:END i=i+1Solve Eq. (4) andmove to next step:Set first traffic record =1k Traffic record loop Structural analysis Image analysis CCTV recording Space-time vehicle curves < ?kNr YesSet first traffic record=1k Traffic record loop Set first vehicle =1v Randomise vehicle weightand assign GWRto each wheel: Pz,w< ?vNv Yes = +1v vNext vehicle: = +1k kNext traffic record:Vehicle loop NoNo Weight randomisationFrameseparation Pixelstatistics Lanedivision Geometrictransform-ation CroppingVehicle identification PwFig. 1.Proposed analysis flowchart. 4 Fig. 2.Pre-processing of a 37-second video captured from the webcam of the old Evripos bridge in Greece. (a) One representative frame of the video showing a series of vehicles. (b) Extraction of the background by removal of all moving objects from the frames by calculating the median value of each pixel along the timedimension. (c)Accumulationofmovementovertimeasthevariationofeach pixel against the background indicates the movement over the bridge. Notice also the movement of the water. imageisusedlaterforthesegmentationofthemovingobjectsbysubtracting the median from any given frame. The standard deviation image highlights the regions of the image where movement of objects occurred. This image is also useful to determine if the recording camera suffers movement (due to e.g. strong wind gusts), which would be revealed by variation of static elements in the field of view. 3.Lanedivision: Thelanesofthebridgearetheregionsofinterestforthestudy, and these are determined from the high intensity pixels of the standard deviation image (Fig. 3(a)). This region is further processed to find the centrelineofthedeckseparatingthetraffic,andtheorientationofthebridge (Fig. 3(b,c)) . 4.Geometric transformation: A geometric transformation [34] based on the previously determined orientation of the bridge is applied to the image to compensate for the perspective provided by the camera. In particular, this studyappliesaforwardtwo-dimensional(2D)projectivewarp[35]withthe followingtransformationmatrix:1−0.031−0.0011 0.194 1 0 .001 0 0 1 . Thistransformation compensatesfortheorientationandchangingwidthoftheviewofthebridge providedbythecamera,anditresultsina trueviewwithahorizontaldeckof constantwidth. Themedianandstandarddeviationimagesaretransformed in the same way and used in subsequent steps. 5 5.Cropping: The transformed image is cropped to focus on the region of interest over the bridge (Fig. 4). 6.Vehicle identification: The segmentation is performed by subtracting the averagevalueofthered,greenandbluechannels(RGB)ofthemedianimage from every frame. Then, the intensity of the output is scaled to the range [0−1]andthresholdingusingOtsu’smethod[36]. Morphologicaloperators [37] (opening, labelling, closing) are employed to clean the objects, with a minimumsizeofobjectbeingappliedtodiscardsmallartefacts. Aparticular challenge in the image processing of traffic occurs when tall vehicles like trucks appear as they could span both lanes of the bridge. These cases are identified by detecting large objects in each lane, when the horizontal span of each of these objects overlapped by more than 50% they are considered to be the same and only one was assigned. Pedestrians are excluded from theidentificationthroughtheuseofhand-drawnmasksthatconcentratedon the two lanes (moving left or right). Fig. 3.Segmentation of the region of interest and extraction of orientation. (a) Segmentation of the lanes of the bridge. (b) Medial line from the segmentation. (c) Straight line adjusted to the Medial line. This line will be used to apply an affine transformation to all frames of the video. Snippets of video from the webcam were saved to a local drive using the software QuickTime (support.apple.com/quicktime). Initial investigations were performed on a 37 second video recorded on the 25 July 2017, and a video of 𝑡𝑎=785s recorded on the 29 November 2021 was then used for the subsequent workinthispaper. ArepresentativeimageofthesequenceisillustratedinFig.2(a). The output of the segmentation process provided one signal per object detected, i.e., a single vehicle would provide one signal at each frame it was detected. The fields of the vehicle signal consisted of: longitudinal position (along-drive, 𝑋) of the vehicle over the bridge in metres, lane of the bridge in which it is located to 6 Fig. 4.Illustration of the transformation and segmentation. (left) One represen- tative frame of the video, which shows one car and two motorcycles in the region of interest. (top) The same frame after warping and cropping. (bottom) Three segmented objects, each indicated by a different colour. obtainthetransverseposition( 𝑌,inmetres),timeinsecondsandareaofthevehicle in pixels. The space-time record of the whole traffic in the bridge is illustrated in Fig. 5 as(𝑋𝑣,𝑡)curves of the longitudinal position of the 𝑣th-vehicle along the deck (horizontal axis) versus the time (vertical axis going down). The slope of these curves gives the speed of each vehicle, which is limited to 30 km/h on the bridge. The two lanes of the deck hold traffic in opposite directions. We consider positive movement the one that goes from left to right, with the origin of the system of coordinates located at the joint of the left abutment ( 𝑋=0), as shown in Fig. 6. Given that the position of the webcam and detail of the video was not sufficient to distinguish the transverse location of the vehicles accurately, it is assumed that they are centred on their lanes, with eccentricity 𝑌=∓1.75m with respect to the bridge centreline ( 𝑌=0) in the Lanes 1 and 2 described in Fig. 6, respectively. All the code related to the image analysis was performed in the programming environment of Matlab®(The MathworksTM, Natick, USA) and is freely available through GitHub: https://github.com/reyesaldasoro/Bridges. Unfortunately, the application of this image analysis process does not give information about the weight of each vehicle crossing the bridge, nor the Ground Wheel Reactions (GWR) that are needed in the structural analysis. The latter can only be determined if weight-in-motion (WIM) is available. In its absence, we propose a randomisation of the weight of the vehicles following the scheme presentedinFig. 1. TheprocesscreatesfromasingleCCTVrecording 𝑁𝑟random 7 Fig. 5.Space-time vehicle curves obtained from the video, with each dot repre- senting one object detected. (a) 4657 objects moving towards the right in Lane 1 grouped as 153 unique vehicles with single trajectory. (b) 7447 objects travelling towards the left in Lane 2 grouped as 212 unique vehicles with single trajectory. (c,d) Zoom in to a shorter time frame to illustrate the movement of the vehicles, each of which is denoted by a random colour for visualisation purposes. trafficrecordsinwhichthespace-timecurvesofeachvehicleisidentical,buttheir weight is obtained from a Gaussian distribution based on their approximate size, given by the average number of pixels associated with them. Based on the identified traffic on the bridge, the flow is categorised into three different types of vehicles to simplify the randomisation of their weights, namely cars,vans,andlightgoodvehicles(LGV).Thedistinctionbetweenthemfromthe CCTV frames is based on the average number of pixels (Pix) of each identified object as it crosses the bridge. Table 1 gives the lower and upper limits of Pix for each vehicle category (Pixsmalland Pixlarge, respectively). During the 785-s CCTV recording a total of 153 and 212 vehicles were identified in lanes 1 and 2, respectively, and according to the proposed categorisation these were distributed in 88 cars, 64 vans and 1 LGV in Lane 1, with 197 cars and 15 vans in Lane 2. It is noted that the moving objects with Pix <500refer to motorbikes and they are ignoredintheanalysis. Forconvenience,thewheelloadsinthestructuralanalysis are referred to the rear of the vehicle. Forsimplicity,thedimensionsofthevehiclesandthedistributionoftheweight 8 3 mPermanent formwork# 6 cm Transverse beams@ 3m: Longitudinalbeam:Concrete slab # 25 cm Asphalt# 10 cm Concretefootway# 20 cm3 rail parapet;posts @ 3 m 1.75 mSidewalk 2 Sidewalk 1 1670mm40mm 40mm680mm 1250mm20mm720mm15mm 15mm250mm 250mm12mm40 mLane 2 Lane 1 xzy Vehicle eccentricity1.75 m 1.6 m 5 m0.25 m 1.75 mSidewalk 1 Lane 10.6 m2 m0.5 m 3.5 m0.06 m 0.2 mCLDeck cross-section: 1.75 mFig. 6.Structural layout and dimensions of the proposed bridge. 9 Car Van LGV HGV Elevation 2.8 4.650.9 2.8 4.40.8 4.27 8.531.7 4.27 15.859.14 1 Plan: GWR 0.2W 0.2W0.3W 0.3W 1.6 0.3W 0.3W 0.2W1.750.2W 0.4W 0.4W0.1W 0.1W2.05 0.22W 0.22W0.22W 0.22W2.050.06W 0.06W Pixsmall500 1500 2200 - Pixlarge1500 2200 4000 - Model small Nissan Micra Ford Transit Courier Isuzu FRR 500 - Model large Audi A8 Ford Transit Custom Iveco 180E25 - 𝑎𝑘 𝑒𝑟𝑏 0.008 0.023 0.027 - 𝑏𝑘𝑒𝑟𝑏 7.7 -23.59 -23.90 - 𝑎max, 𝑝𝑎𝑦𝑙𝑜𝑎𝑑 0.002 0.030 0.039 - 𝑏max,𝑝𝑎𝑦𝑙𝑜𝑎𝑑 2.7 -37.94 -62.78 - 𝑊min, 𝑝𝑎𝑦𝑙𝑜𝑎𝑑 0.5 0.5 0.5 - T able 1.Geometry and dimensions of the vehicles considered in the structural analysis. Pixsmalland Pixlargerefer to the lower and upper limits of the pixel band for the corresponding vehicle. Note: the large HGV is based on the AASHTO HS20-44 truck [38] and it was not identified in the image analysis, instead, it is proposed for code-based calculations in Section 3. Dimensions in meters and weights in kN. amongtheirfourwheelsareonlyconsideredafunctionoftheircategory(car,van or LGV), and not their size (Pix). However, the latter affects the total weight of thevehicle𝑊,whichisthesumofitskerb(empty)weight, 𝑊𝑘𝑒𝑟𝑏,anditspayload (cargo),𝑊𝑝𝑎𝑦𝑙𝑜𝑎𝑑, and it is different in each vehicle of the recorded traffic flow. Thedefinitionoftheweightisbasedondataprovidedbymanufacturersofvehicles takenasreferenceforthesmallandlargebandsofeachvehiclecategory,included in Table 1. The kerb weight is given as 𝑊𝑘𝑒𝑟𝑏=𝑎𝑘𝑒𝑟𝑏Pix+𝑏𝑘𝑒𝑟𝑏, (1) in which𝑎𝑘𝑒𝑟𝑏and𝑏𝑘𝑒𝑟𝑏are the slope and the ordinate-intercept of the linear equationtheweproposetodefine 𝑊𝑘𝑒𝑟𝑏,respectively,andtheyaregiveninTable 1 for the three different types of vehicles considered. 10 The randomisation of the traffic flow comes from the definition of the vehicle payload (𝑊𝑝𝑎𝑦𝑙𝑜𝑎𝑑), which is the most uncertain value given that we can only estimate its size from the CCTV recording, but not its contents. This weight is taken from a normal probability distribution N(𝜇𝑝𝑎𝑦𝑙𝑜𝑎𝑑,𝜎2 𝑝𝑎𝑦𝑙𝑜𝑎𝑑)centred in the mean value𝜇𝑝𝑎𝑦𝑙𝑜𝑎𝑑with a standard deviation 𝜎𝑝𝑎𝑦𝑙𝑜𝑎𝑑, and it is limited between a minimum and a maximum payload as 𝑊𝑝𝑎𝑦𝑙𝑜𝑎𝑑=maxn minh 𝑊max,𝑝𝑎𝑦𝑙𝑜𝑎𝑑,N 𝜇𝑝𝑎𝑦𝑙𝑜𝑎𝑑,𝜎2 𝑝𝑎𝑦𝑙𝑜𝑎𝑑i ,𝑊min,𝑝𝑎𝑦𝑙𝑜𝑎𝑑o , (2) where𝑊min,𝑝𝑎𝑦𝑙𝑜𝑎𝑑 istheminimumpayloadinthevehicle,whichisconsideredas 0.5 kN regardless of its size to consider an empty vehicle with only a relatively light-weight driver; 𝑊max,𝑝𝑎𝑦𝑙𝑜𝑎𝑑 is the maximum payload and it depends on the vehicle size as 𝑊max,𝑝𝑎𝑦𝑙𝑜𝑎𝑑=𝑎max,𝑝𝑎𝑦𝑙𝑜𝑎𝑑 Pix+𝑏max,𝑝𝑎𝑦𝑙𝑜𝑎𝑑, (3) with𝑎max,𝑝𝑎𝑦𝑙𝑜𝑎𝑑 and𝑏max,𝑝𝑎𝑦𝑙𝑜𝑎𝑑 obtained from the data provided by the man- ufacturers of vehicle models taken as representative in each vehicle category in Table 1. The mean value of the normal payload distribution is considered as the average between the minimum and the maximum values, and the standard devi- ation is assumed to be 20% the corresponding mean value in all the cases. The implications of this choice in the dynamic response of the bridge are discussed later in the paper. For comparison purposes, two additional load cases with convoys of equally- spaced heavy goods vehicles (HGVs) are considered in the subsequent dynamic analysis: (1) HGVs crossing the bridge using only Lane 1, with the other lane completely empty, and (2) HGVs crossing the bridge using both lanes (Lane 1 in positive-𝑋and Lane 2 in negative- 𝑋directions). In both cases the speed of the truck is kept constant at 30 km/h, and the distance between consecutive vehicles is calculated to give a time of 2 s between the rear of one truck and the front of the following, resulting in a 25.3-m spacing between them. The convoys are composedof259truckscrossingthebridgeintheseadditionalloadcasesinorder tohaveacompleteanalysistimeofthesamedurationaswiththeimage-processed flows(𝑡𝑎=785s). TheHGVsinthesesimplifiedloadcasesarebasedonthelarge AASHTO HS20-44 truck [38] that is described in Table 1, with a total weight of 𝑊=320kN. 11 42 13Ww PZ,w,1PZ,w,4PZ,w,3PZ,w,2 xwr rLx ywLyA BFig. 7.FE model of the bridge and load interpolation of a single wheel. 2.2 Structural analysis The proposed bridge is a 40-m span composite ladder-deck with a concrete slab supported by two longitudinal I-shaped steel girders that are connected by trans- verse beams at 3-m intervals, as shown in Fig. 6. The recorded traffic flow is appliedtoafiniteelement(FE)modelofthebridgeinwhichtheslabisdiscretised with728full-integration4-nodeshellelementsofapproximately 0.8×0.8m,and a total of𝑁𝑑=795nodes. The longitudinal and transverse steel beams are mod- elled with linear interpolation beam elements rigidly connected to the concrete slab. The distance between the centre of the supports and the girder end is 0.4 m. The two supports located under Sidewalk 2 restrain the lateral ( 𝑌) motion of the deck, whereas the other two release it, as it is illustrated in Fig. 6. It is assumed that the response of the bridge under the traffic actions is linear and elastic. Therefore, the dynamic response of the structure can be obtained as the superposition of the time-history contribution of a reduced set of 𝐽vibration modes. The participation of the 𝑗-th mode is given by the coordinate 𝑞𝑗and its time-derivatives (¤𝑞𝑗and¥𝑞𝑗) as ¥𝑞𝑗(𝑡)+2𝜉𝑗𝜔𝑗¤𝑞𝑗(𝑡)+𝜔2 𝑗𝑞𝑗(𝑡)=𝜙𝜙𝜙T 𝑗P(𝑡) 𝑚𝑗, (4) where𝜙𝜙𝜙𝑗,𝜔𝑗,𝜉𝑗and𝑚𝑗refer to the shape, circular frequency, damping ratio and mass of the 𝑗-th mode, respectively. The accurate representation of the vibration modes requires great detail in the FE model to capture both the stiffness and the massofthestructure,includingthestructuralmassandthenon-structuralonegiven by the asphalt (modelled as increased concrete density), parapets and sidewalks (represented with lumped masses at their corresponding positions). However, 12 these details of the model are not explicitly included in the dynamic analysis of the traffic-induced vibrations. Instead, substructuring is proposed to speed up the dynamic analysis by considering only the modal displacements 𝜙𝜙𝜙𝑗corresponding to the slab of the deck, ignoring the degrees of freedom associated with the steel beams. This is possible because they are not directly loaded by the vehicles and thereforetheydonotcontributetothenodalforcingvector P(𝑡). Forconvenience, in this work Pis organised as follows P(𝑡)={0|{z} X-forces,0|{z} Y-forces,𝑃𝑍,1(𝑡),···,𝑃𝑍,𝑁 𝑑(𝑡)| {z } Z-forces}T, (5) where𝑃𝑍,𝑚(𝑡)istheverticalforceatthe 𝑚-thnodeoftheslabduetothevehicles on the deck at time 𝑡. Horizontal (braking) and lateral wheel forces are ignored in this study (hence 𝑃𝑋,𝑚=𝑃𝑌,𝑚=0at any node 𝑚), as well as the vehicle vibration and its interaction with the bridge. The latter can be included in the analysismethodology,butitrequiresthedefinitionofthemechanicalpropertiesof thedifferenttypesofvehiclescrossingthebridge,anditwoulddiverttheattention from the goal of the study which focuses on the response of bridges (not the vehicles) under image-processed traffic flows. The time-domain analysis requires a nested FOR-loop in which for each time- stepoftheanalysis(outerlayer)thecontributionofeachwheelonthedeck(inner layer) to the nodal forcing vector in Eq. (5) is calculated, as shown in Fig. 1. The following is considered to speed up computation and make the analysis of long traffic flows feasible: •Discard from the analysis the vehicles that are not on the bridge: In Eq. (4) it is only necessary to consider the vehicles with at least one wheel axle on the deck at a given time 𝑡𝑖of the analysis. Therefore, in this time-step the wheel-loop only needs to run over the 𝑁𝑤wheels of the vehicles on the bridge, which is smaller than the total number of vehicles registered in the CCTV image analysis ( 𝑁𝑣). To this end, an existence matrix 𝚲={Λ𝑖,𝑣}is created for the observed traffic flow, prior to the dynamic analysis, with Λ𝑖,𝑣=( 1if at least one axle of vehicle 𝑣is on the bridge at time 𝑡𝑖, 0otherwise.(6) Therefore, at each time 𝑡𝑖only the vehicles with Λ𝑖,𝑣=1are considered in the dynamic analysis. 13 •Accelerate the search of adjacent nodes: One of the processes that is more computationallyexpensiveisthesearchofthenodesadjacenttotheposition of each wheel on the deck. The use of traditional search functions in pro- gramminglanguageshasatime-complexity O(𝑁𝑑),andthecalculationtime increaseslinearlywiththenumberofnodesinwhichitissearched. Instead, we propose a binary search that halves the search interval in each iteration, and it is significantly more efficient, with O(log(𝑁𝑑))[?]. The search is conductedindependentlyinthelongitudinalandtransversedirectionstofind theshellelementoftheconcreteslabthatisdirectlyloadedbyeachwheel 𝑤, whichisfacilitatedbythestructuredmeshshowninFig. 7. Thewheelload is lumped to the 4 adjacent nodes as 𝑃𝑍,𝑤,𝑘𝑙=𝛼𝑘𝑙𝑊𝑤, with𝑘𝑙=1,2,3,4 referringtothelocalnumberingoftheshellelementinFig. 7andthelinear interpolation functions: 𝛼1= 1−𝑥𝑟 𝑤 𝐿𝑥  1−𝑦𝑟 𝑤 𝐿𝑦 , (7a) 𝛼2=𝑥𝑟 𝑤 𝐿𝑥  1−𝑦𝑟 𝑤 𝐿𝑦 , (7b) 𝛼3=𝑥𝑟 𝑤 𝐿𝑥 𝑦𝑟 𝑤 𝐿𝑦 , (7c) 𝛼4= 1−𝑥𝑟 𝑤 𝐿𝑥 𝑦𝑟 𝑤 𝐿𝑦 . (7d) where𝑥𝑟 𝑤and𝑦𝑟 𝑤aretherelativedistancesfromtheloadtothelocalNode1 oftheloadedshellinthelongitudinalandtransversedirections,respectively; 𝐿𝑥and𝐿𝑦arethecorrespondingelementlengths(inthisstudy 𝐿𝑥≈𝐿𝑦≈0.8 m). •Vectorisationofmodaldynamicssolver : thetraditionalmodalsuperposition requiresrepeatingsequentiallythecalculationinEq. (4)forallthevibration modes that are relevant to the response of the structure. This is avoided bycomputingthemodalforcing ˆPwithamatrix-multiplicationofthemode shapematrix 𝚽containingalltherelevanteigenvectors( 𝜙𝜙𝜙𝑗)timesthelumped nodal loads given by all the wheels on the deck at the time 𝑡𝑖 ˆP=𝚽TP, (8) 14 in which it has been assumed that the modal shapes are mass-normalised. Thearraywiththemodalcoordinatesofalltherelevantvibrationmodes(q) canbeobtainedbyintroducingthemodalforcing ˆPintheright-handsideof Eq. (4) and solving it simultaneously with the non-iterative and vectorised Newmark-𝛽methoddescribedin[39]. Finally,theresponseofthestructure at time𝑡𝑖and its time-derivatives are obtained directly as: 𝑟(𝑡𝑖)=𝚽q(𝑡𝑖);¤𝑟(𝑡𝑖)=𝚽¤q(𝑡𝑖);¥𝑟(𝑡𝑖)=𝚽¥q(𝑡𝑖). (9) The FE model of the structure was conducted in the software Abaqus [?] to obtain the vibration modes. The most relevant ones are included in Fig. 8. The fundamental mode is a global vertical flexure of the deck ( 𝑓1=2.01Hz) and the second one describes its global torsion ( 𝑓2=2.88Hz). Due to the large width- to-span ratio of the bridge there are high-order vibration modes with transverse flexure of the slab that fall in the range between 18 and 50 Hz. These are referred to as slab modes and it will be demonstrated later that they are important for the acceleration in the sidewalks. Therefore, the first 𝐽=60vibration modes are considered in the dynamic analysis, and the time-step is set as Δ𝑡=0.002s to capture accurately all the relevant modes. It is noted that the analysis time-step is smallerthanthatwithwhichtheCCTVrecordinginthebridgeisprocessed(0.25 s), therefore linear interpolation was applied to find the position of the vehicles. Foreachrandomtrafficrecord 𝑘thedynamicresponseofthebridgeisobtained by implementing the proposed methodology in the Python library MDyn [39]. The damping ratio is considered 𝜉=0.5% for all the vibration modes, following [7, 8, 40]. The total time of the analysis in each record is 𝑡𝑎=785s, which coincides with the duration of the CCTV image processing. A total of 𝑁𝑘=100 different records of the traffic loading are considered. 3 Results Fig. 9 shows a representative time interval of the dynamic vertical displacement of the bridge at midspan (sidewalk edge points A and B in Fig. 7) when the structure is subject to the randomised image-processed traffic flows, with cars, vans and LGVs crossing the structure in both traffic lanes. The results in this figurerepresentthemeanofthe100randomisedtrafficflowrecordsobtainedfrom the image analysis. In addition, the displacements obtained with the simplified load cases based on convoys of AASHTO trucks are included in the figure. The 15 (a)Mode 1, f1 = 2.01 Hz (b)Mode 2, f2 = 2.88 Hz (c)Mode 15, f15 = 18.19 Hz (d)Mode 34, f34 = 34.38 Hz (e)Mode 44, f44 = 41.23 Hz Slab modesFig. 8.Relevant vibration modes of the bridge and their natural frequencies 𝑓. candence of the passing trucks in the convoy load cases is clear from the results, with an excitation interval of approximately 3 s (0.33 Hz) given by the constant spacing between trucks and their speed. This contrasts with the more random response of the bridge under the recorded traffic, which also gives significantly lower displacements due to the smaller loads compared with those of large HGV trucks. As expected, the peak displacements are similar in both sidewalks in the loadcasesforwhichbothlanesareoccupied,butforthecasewithtrucksinlane1 exclusively the general torsion of the bridge is activated and the displacements at Point B (i.e. in the sidewalk next to the loaded lane) are larger. Therecord-to-recordvariabilityoftherandomisedimage-processedsignalsin the vertical displacements at the same two points at midspan is presented in Fig. 10 for a narrower time window. It is observed that a variance of 𝜎2=0.04𝜇2 in the vehicles’ mean cargo ( 𝜇) introduces a variability in the peak dynamic displacementsatmidspanofupto22%. Morespecifically,thepeakdisplacement at midspan at Point B range from 1.53 to 1.62 mm in the 100 records considered, and it goes from 1.46 to 1.88 mm at point A. The peak displacement of 1.88 mm occursforrecord#40,whichwillbeconsideredfurtherinthediscussionasitgives the largest response of the bridge. The dynamic effects of the moving vehicles on the bridge are obtained by comparingthedynamicdisplacementsobtainedfromEq. (4)andtheirpurelystatic 16 (a) (b) Fig. 9.Time-historyoftheverticaldisplacementatmidspan: (a)pointA,(b)point B. Individual record Individual record(a) (b) Fig. 10.Detail of the time-history of the vertical displacement at midspan due to the recorded traffic: (a) point A, (b) point B. 17 counterparts. Thelatterareobtainedfromthesametrafficflowsbycancellingthe damping and inertia terms in Eq. (4): 𝑞𝑠𝑡 𝑗(𝑡𝑖)=𝜙𝜙𝜙T 𝑗P(𝑡𝑖) 𝜔2 𝑗, (10) where𝑞𝑠𝑡 𝑗is the static coordinate of mode 𝑗, and it is assumed that the vibration modeshapesaremass-normalised. Afterasensitivityanalysistodefinethenumber ofmodesconsideredinthestaticanalysis,thetotalstaticresponse( 𝑟𝑠𝑡)isobtained as the superposition of the contribution of the first 𝐽=60modes: 𝑟𝑠𝑡(𝑡𝑖)=𝚽qst(𝑡𝑖). (11) Fig. 11comparestheverticalstaticanddynamicbridgedisplacementsatpoint Bundertherecordedtrafficflowwithlargestresponse(record#40),andalsounder thetwoloadcaseswithconvoysofAASHTOtrucks. Itisapparentthattheinertia forces increase the displacements of the bridge, for which the dynamic response presents a characteristic frequency of approximately 3 Hz that is related to the secondvibrationmodeofthebridgeshowninFig. 8,asitwillbeexploredinmore detail later. The dynamic impact factor at the 𝑚-th node of the deck slab is defined as IF𝑚=max 𝑡𝑖[|𝑟𝑚(𝑡𝑖)|] max 𝑡𝑖[|𝑟𝑠𝑡𝑚(𝑡𝑖)|], (12) in which𝑟𝑚and𝑟𝑠𝑡 𝑚are the dynamic and static displacements at node 𝑚, re- spectively. The IF is included in Fig. 12 for all the nodes along the edges of both sidewalks, and it is compared with the value recommended by AASHTO to account for traffic-induced dynamic effects in the static analysis of road bridges (IF=1.33). The typical values of the IF obtained in this bridge are below this limit and oscillate around 1.15. However, close to the supports of the deck the vertical static displacements are very small and make the definition of the IF in Eq. (12) more unreliable. For the same reason, the IF of the response along sidewalk 2 is higher in the load case with a convoy of trucks concentrated in lane 1, because the torsion generated reduces the vertical displacement in the opposite sidewalk. Nevertheless, it is interesting to note that despite the larger weight of the vehicles crossing the bridge in the two load cases with HGV convoys, the dynamicamplificationofthedisplacementsinmostofthelengthofSidewalk1is 18 Fig. 11.Comparison between the dynamic and the static responses at point B of the deck, for different load cases. 19 (a) (b) AASHTO IF AASHTO IF Fig. 12. Dynamic impact factor along the sidewalk edges of the bridge: (a) sidewalk 2, (b) sidewalk 1. similar to that observed with the image-processed traffic. In addition, the record- to-recordvariabilityoftheIFobtainedwiththelatterisrelativelysmall(below4% atmidspan),whichsuggeststhatconsideringthemeanvalueofthevehicleweight distribution gives a reasonable value of the dynamic traffic-induced effects in this type of bridges. 3.1 Comfort analysis Theassessmentofthepedestrians’comfortonthesidewalksofthebridgedepends onthe frequencycontentof itsvibrations (e.g. [16, 19,41]). Theevolution ofthe contributionofdifferentfrequenciestotheverticaldisplacementsandaccelerations atpointA(sidewalk2)astheimage-processedtrafficrecord#40excitesthebridge is presented in the spectrograms of Figs. 13(a) and (b), respectively. The results in Fig. 13(a) indicate that the displacement response at midspan is dominated by thelow-ordervibrationmodes1and2(seeFig. 8),whereastheslabmodesabove 18Hzwithlocalflangedeformationatthesidewalkareashavelowercontribution, particularly in the intervals with less recorded traffic on the bridge at 𝑡∼50s, 220 s and 360 s, as shown in Fig. 5. The displacement signal also captures the static effect of the vehicle entrance in the bridge, with an average frequency of 0.46 Hz (given by the access of 365 vehicles to the deck in 785 s). However, the pedestrians’ comfort depends on the accelerations, and Fig. 13(b) indicates that 20 (a) Mode 34 Mode 15 Mode 2 Mode 1Avg. vehicle entrance frequencyMode 34 Mode 15 Mode 2 Mode 1 (b)Fig. 13.Spectrogramoftheverticalresponseofthebridgeatmidspanofsidewalk 2 (point A) due to the recorded traffic (record #40): (a) displacements, (b) accel- erations. these are more influenced by high-order slab modes, especially modes 15 and 34, which is in agreement with previous works on bridges with large width-to-span ratios [14, 19]. The effect of the gaps in the actual vehicle flow recorded by CCTV make the frequencycontenttime-dependent. Thiscontrastswiththeuniformbandsofmodal contributionwhenthebridgeissubjecttothecode-basedconvoysofheavytrucks shown in Fig. 14. These load cases exaggerate the participation of high-order modesabove20Hz,particularlymodes34and44includedinFig. 8. Therefore,it seems important to consider more realistic vehicle flows in the dynamic analysis. We use Irwin’s criterion [42] to assess the comfort of pedestrians standing on the sidewalks of the bridge. To this end, the root-mean-square (RMS) of the acceleration signal ¥𝑟(𝑡)is calculated at different one-third octave bands as: ¥𝑟RMS(𝑓𝑐)=√︄∫𝑓𝑢 𝑓𝑙𝑆¥𝑟¥𝑟d𝑓, (13) inwhich𝑆¥𝑟¥𝑟isthePowerSpectralDensity(PSD)oftheaccelerationsignal ¥𝑟(𝑡);𝑓𝑙 and𝑓𝑢are the lower and the upper frequencies of each octave band, respectively. 21 (a) Mode 34 Mode 15 Mode 2 Mode 1Mode 34 Mode 15 Mode 2(b) Mode 1Mode 44 Mode 44Fig. 14.Spectrogramoftheverticalresponseofthebridgeatmidspanofsidewalk 2 (point A) due to the AASHTO truck loading: (a) trucks in one lane, (b) trucks in two lanes. The RMS acceleration ( ¥𝑟RMS) at each octave band is allocated to their central frequency𝑓𝑐, for which the lower and upper frequencies are 𝑓𝑙=2−1/6𝑓𝑐and 𝑓𝑢=21/6𝑓𝑐, respectively. Finally, the calculated ¥𝑟RMSis compared with the frequency-dependent comfort limit suggested by Irwin for frequent conditions [42] in Fig. 15, for different load cases. It is observed that the vibrations at the midspan section induced by the convoys of heavy trucks are categorised as uncomfortable, in both sidewalks. This is due to the effect of the first global torsional mode of the bridge (mode 2) and the high-order slab modes, and it is particularly large in sidewalk 2 due to the laterally-restrained support conditions at this side of the bridge. However, the comfort limits are not exceeded under any of the randomised image-processed traffic flows, for which mode 1 gives the RMS acceleration that is closest to the discomfort threshold. The recorded trafficcasesleadtoasignificantlylowerparticipationofthehigh-orderslabmodes in the pedestrians’ sense of vibrations, compared with that obtained under the long convoys of equally-spaced trucks. In addition, the latter load cases present unrealistic peaks of ¥𝑟RMSfor frequencies significantly lower than that of the first bridge mode: 𝑓𝑙1=0.33Hz and𝑓𝑙2=2𝑓𝑙1=0.66Hz, which are given by the artificial excitation frequency of the convoy of trucks. 22 (a) (b)Mode 1 Mode 2 Mode 15 20 3040 20 3040Mode 34 Mode 44 Mode 1 Mode 2 Mode 15 Mode 34 Mode 44 fl1 fl2 fl1 fl2 Fig. 15.Vertical RMS accelerations at midspan for different load cases: (a) point A in sidewalk 2, (b) point B in sidewalk 1. Note that the image analysis presents theresultsof100randomrecordsingreycolour,buttheyarealmostsuperimposed to their mean value, depicted in red colour. Finally, the comfort assessment is extended to the entire length of both side- walks by means of a discomfort risk ratio 𝜂 𝜂𝑚=max 𝑓¥𝑟RMS,𝑚(𝑓) ¥𝑟𝑙𝑖𝑚(𝑓) , (14) in which the subindex 𝑚refers to the node of the sidewalk where it is calculated, and¥𝑟𝑙𝑖𝑚(𝑓)is the maximum admissible RMS acceleration proposed by Irwin in frequentconditions(seeFig. 15). Inaddition,vibrationsareassessedbasedonthe peak accelerations along the sidewalks considering the BS5400 comfort criteria [43]: 𝜂𝑚=max 𝑡[|¥𝑟𝑚(𝑡)|] ¥𝑟𝑙𝑖𝑚, (15) with¥𝑟𝑙𝑖𝑚=0.5√︁ 𝑓1= 0.71 m/s2being the maximum admissible peak acceleration givenin[43],and 𝑓1=2.01Hzisthefundamentalverticalfrequencyofthebridge. The discomfort risk ratios obtained for both comfort criteria along the side- walks are included in Fig. 16, considering the accelerations induced by the 100 differentimage-processedtrafficflows. Therecord-to-recordvariabilitywithinthe randomised traffic flows is significant (up to 33%) in terms of the peak accelera- tions,whichexceedtheBS5400comfortlimitinmostofthebridgesidewalkareas, 23 UncomfortableUncomfortable(a) (b) BS5400 criterion Irwin's criterion Irwin's criterionBS5400 criterion Fig. 16.Discomfortriskratioalongthesidewalkedgesofthebridgeunderallthe imageanalysisrecords: (a)sidewalk2,(b)sidewalk1. Thethingreylinesreferto theresultsforeachindividualtrafficrecord,thethickerredlineistheirarithmetic mean, and the coloured band around it represents one standard deviation to each side of the mean value. particularlyinthesidewalk2duetothelateralconstrainofthesupportsatthatside of the bridge. However, considering a more accurate comfort criteria based on the RMS accelerations reduces significantly the influence of the vehicle payload weight variability, with a record-to-record variation below 2%, and it makes it possibletoconsiderasingletrafficrecordwithmeanvehicleweightdistributions. Inaddition,theRMS-basedcomfortcriterionissatisfiedintheentirebridgeunder the recorded traffic flows. 4 Conclusions ThisworkpresentsamethodologytocombineCCTVimageprocessingandmodal structural dynamics to analyse the traffic-induced vibrations in bridges, without the need for monitoring or weight-in-motion data. The image analysis techniques are used to identify the motion of vehicles on a real bridge in Greece, and a randomisation process is proposed to generate records with different plausible weights based on the size of the observed vehicles. The work follows introducing a method to define the modal forcing in shell-like finite element discretisations of the deck, which is applied to the model of a conventional ladder-deck composite bridge. Thedynamicanalysisofthestructureunderasignificantnumberoftraffic 24 flows and convoy-based load cases led to the following observations: •A standard deviation of the vehicle payload fixed as 20% of their mean value results in record-to-record differences of up to 22% and 33% in the peak displacements and accelerations at midspan, respectively. However, the variability is reduced below 4% and 2% in the dynamic impact factor andtheRMSaccelerationsatthesamesection,makingitpossibletorunthe dynamicanalysisforasingletrafficrecordwithassumedmeanvaluesofthe vehicles’ payload, in the absence of weight-in-motion data. •Theunrealisticloadcasescomposedonlyofheavytruckscrossingthebridge equallyspacedandwithconstantspeedexaggeratethecontributionofhigh- order vibration modes with local flexure of the slab at the flanges where the sidewalks are located, leading to vibrations that are categorised as un- comfortable. However, the comfort criterion based on RMS accelerations is satisfied in the entire length of the sidewalks with all the recorded traffic flows. •It is observed that the transverse flexure of the slab and the boundary con- ditions of the supports in this direction are important for the vibrations in the sidewalks. Therefore, to capture these effects it is recommended to discretise the upper slab with shell elements, which can be separated for efficiency from the rest of the structural and non-structural members of the finite element model in the modal analysis, as proposed in the paper. This work aimed at establishing a framework where new image processing andartificialintelligencetechniquescanbeappliedinfurtherstudiestoanalysein real time the response of bridges from CCTV recording, therefore assisting in the assessment of discomfort and fatigue risks in structures. 5 Acknowledgements The authors would like to express their gratitude to Dr. Panagiotis Mergos for his valuablecommentsandtheinitialguidancetofindCCTVbridgemonitoringdata. 25