Long-term monitoring of traffic loads across the whole bridge is of great significance for bridge health monitoring a nd safety assessment, especially for long- span and complex bridges. However, none of the existing schemes have operated long term, and the full-bridge traffic load (FBTL) monitoring has remained a concept. In this paper, an improved FBTL monitoring framew ork capable of long-term operation was proposed, through fusion of the weigh-in-m otion system (WIMs) and multi-camera vision systems. In the system, vehicle dete ction and tracking algor ithms with bettter accuracy and robustness were achieved with deep convolutional neural networks, which could effectively deal with the pr oblem of target loss during long-term monitoring. A fast but accurate correction method for transverse vehicle position was proposed by applying projective geometry, en suring the system could operate with a large number of target vehicles. The proposed FBTL monitoring system was successful in generating the traffic load distribu tion of a long-span cable-stayed bridge, demonstrating its technical feasibil ity and engineering applicability. Key words: full-bridge traffic load, long-term monitoring, vehicle detection, vehicle tracking, fast vehicle position correction, long-span bridge 1 Introduction Traffic load is one of the dominant live loads on bridges and th e resulting structural response is a direct reflection of bridge s’ safety and operating condition. It is necessary to take traffic lo ads into consideration throughou t the life-cycle of bridges, including bridge design, heal th monitoring, maintenance and even restrengthening. In particular, in the field of bridge health monitoring, an accurate measurement of FBTL is of great importance for challenging problem s such as load effect assessment [1-2], model updating [3-4] and damage identification [5-6]. Due to the technical complexity of the di rect FBTL measurements, three indirect approaches have been investigated. (1) The first approach was using the inversi on of structural responses, known as the BWIM technique [7-9]. It was shown that the BWIM method can estimate simple forms of loads (e.g, concentr ated load) if an accurate baseline physical model is available [8-9]. However, it is difficult to accurately determine the baseline model of a long-span bridge. Also, it is ch allenging to achieve load inversion mathematically for complex traffic situati ons with multiple vehicles (loads) at the same time. (2) The second approach is using the pavement -based weigh-in-motion system (WIMs) with certain assumptions such as no overtaking and lane changing. Using WIMs data, Obrien et al. [10] and Zhou et al. [11] applied traffic flow theory to simulate the FBTL on different types of bridges. Th ese studies verified the load simulation with the reasonable assumption that this would be useful for the long-term assessment of bridges. However, due to th e difference from the actual situation, the FBTL simulation sometimes could not reproduc e the real conditions of the bridges. For example, in the load analysis of the Yongjiang Bridge, Ge et al. [12] found that the WIMs data only would cause statistical underestimation of the traffic load effect. (3) The third approach is to obtain complete traffic load information through a field load test, which uses controlled vehicl es positioned in specific locations on the bridge. This method has been applied in many studies of model updating and been proven very effective for different types of bridge [13-15]. Howe ver, the load test inevitably requires an interruption of traffic, while providing a limited record of measurements due to the short test time. These traffic load distributions from the indi rect approaches are either quite different from the actual situation (especially for long- span bridges), or not enough to be used for long-term bridge health monitoring. With the rapid development of sensors and computer vision technology, extensive studies have emerged in recent years for di rect traffic load measurements, combining data from multiple types of sensors, of which the most common is the vision-based method [16-21]. The study of vision-based met hods started with partial observation of traffic loads: Ryan and Al [16] applied the background difference method to identify the vehicle position on the bridge and presented a detailed discussion on the identification accuracy. Furthermore, adding in formation about vehicle weight, Chen et al.[17] employed the same method to obtain the vehicle position in mutiple localised areas on a cable-stayed bridge and estimated the FBTL distribution, under the assumption that the vehicles did not cha nge lanes. To reproduce the traffic load distribution, Dan and Ge [18] started a new monitoring scheme for the complete FBTL observation by fusion of the WIMs and multip le cameras covering the whole bridge deck. Since then, some researchers have used deep learning techni ques to improve this scheme. Zhou et al. [19] proposed a novel vehicle detection method based on the faster region-based convolutional neural network (Faster R-CNN), effectively improving the system robustness to ambient lighting. Furt hermore, Ge et al. [20] proposed an improved method for better vehicle transv erse positioning accuracy using the dual target detection and vehicle dimension estimation. This improvement showed good performance in terms of ligh ting robustness, accuracy a nd calculation speed. Even though the framework of full-bridge traffic load (FBTL) monitoring has been improved significantly, there are still chal lenges for long-term FBTL monitoring systems, as outlined below. (1) In real applications of the system, it is more convenient to install cameras on the side of the road. This introduces a large tr ansverse positioning error, which can be accurately corrected by [20]. However, the correction takes a relatively longer time, especially for multi-lane long-span bridges. (2) Occlusion is an obstacle for successful ve hicle tracking, effectively detaching the weight information from the detected ve hicle. A robust vehicle tracking method is required to ensure continuous vehicle tracking. This paper proposes a long-term FBTL monitoring framework and improved methods for better performance and applicability. The contributions of this paper are: (1) For the first time, FBTL monitoring has been realised on a long-span bridge, Yongjiang Bridge. (2) A YOLO-v4 based vehicle detection model was shown to detect distant small vehicles, which was better for the real image dataset from Yongjiang Bridge. (3) A robust vehicle tracking strategy is propos ed to address the problem of vehicle occlusion, combining the vehicle moti on with the appearance features. (4) A fast correction method for transverse vehicle position is proposed, using regression on the correction factor. This paper is organised as follows. Sec tion 2 introduces the overall framework of the proposed FBTL monitoring system . Section 3 decribes the hardware implementation of the FBTL on Yongjiang Brid ge. Sections 4-6 describe the improved deep learning vehicle detection model, the robust vehicle tracking method, and the fast position correction method, respectively. Sect ion 7 describes the transformation from the image to the bridge coordiates. Section 8 shows the results of the proposed system to Yongjiang Bridge, and Secti on 9 presents the conclusion. 2 Overall framework for the long-t erm FBTL monitoring system Full-bridge traffic load (FBTL) monitoring re fers to the identification of the traffic load distribution on the whole bridge deck in terms of vehicle weight and position. Figure 1 is the flowchart of the proposed long-term FBTL monitoring system, covering the six key steps, the corresponding preparations on hardware and software, and three core tasks, as introdu ced in detail in this paper. The required inputs to the proposed system are the segments of WIM text data and videos from the same period of time, while the output is the spatial distribution of traffic loads on the whole bridge deck at the corr esponding moment. In general, the videos contain the position information of all passi ng vehicles, while the WIM text data records the information of vehicle weight , speed, lane number and crossing time. According to the order of input to output, the execution steps of the long-term FBTL monitoring system are as follows. Step 1: First frame detection and weight matching The sensors required for the FBTL monitoring include two WIM systems and multiple high-definition (HD) cameras. The WIM systems are installed at both entrances of the bridge to obtain the inform ation on entering vehicles, while the cameras are arranged along the bridge to record videos with overlappi ng fields of view (FOVs). Vehicles entering the bridge are recorded by both the WIM system and the first camera for each direction. The weight information from the WIMs is attached to the vehicle detected in the video until th e vehicle leaves the bridge. Step 2 : Vehicle detection in each field of view (FOV) For the video captured in each FOV , the ve hicle detection is carried out frame by frame to get vehicle positions in the image coordinates. The core task of this step is to design a high-performance vision-based ob ject detection model. To meet the requirements of long-term monitoring on long- span bridges, this model needs to be robust against environmental changes and capab le of detecting small targets far from the camera. Figure 1. Flowchart of the long-term FBTL monitoring system Step 3 : Continuous vehicle tracking in each FOV Vehicle tracking refers to the pract ice of continuously ma tching the detection results in adjacent frames to obtain vehicle tr ajectories over a period of time. This step inolves conducting continuous tracking of the vehicle in each FOV , while reducing trajectory loss, even if pa rtial occlusion occurs. A novel dual-metric tracking method combining the vehicle motion and ap pearance features is proposed. Step 4 : Handover of vehicle informa tion between multiple FOVs To obtain the full-bridge traffic load distribution, it is necessary to establish the correspondence between vehicles captured by two adjacent cameras and ensure the transfer and sharing of vehicle information across the two FOVs. This step will realise it by applying the visibility of vehicles in different fields of view, i.e. the FOV line method proposed in [18]. Step 5：Correction of vehicle image coordinates From a practical point of view, it is easier to install cameras on the side of the road along the bridge. However, this introduces a large error in the identification of vehicle transverse positions [20]. This step invol ves making a correction using a quick method presented in Section 6. Step 6 : Transformation from image coordina tes to bridge deck coordinates For each camera, the transformation relati onship from the image coordinates to bridge deck coordinates is different. Diff erent transformation matrices of the cameras are calculated through the pr ojective geometry method [18] and applied to convert all image coordinates into th e bridge coordinates. After the above steps, the FBTL monito ring system can provide the full-bridge traffic load distribution at the current frame. For the next calculat ion frame, repeat the above 6 steps until the calcul ation of input segements is done. Throughout the process, vehicle detection, track ing and position correction are three key tasks. To complete these tasks properly, some preliminary prep arations are also required, such as time synchronisation, image dataset making and camera parameter calibration. In the following sections, these key tasks and prep arations are discussed for the FBTL monitoring system developed using Yongjiang Bridge, Ningbo, China. 3 Hardware of the FBTL monitori ng system on Yongjiang Bridge In this study, the long-term FBTL monito ring system is developed using Yongjiang Bridge, a single-tower cable-st ayed bridge, which is about 200 meters in length. This bridge carries heavy urban traffic, equi pped with a complete structural health monitoring system, including sensors such as WIMs (2), cameras (4), GPS (1), accelerometers (22), displacement gauges (4) and strain gauges (16), (Figure 2.) Figure 2. The FBTL monitoring hardware layout on Yongjiang Bridge Figure 3 shows the HD cameras and WIM systems. Two WIM systems (MEAS quratz sensor of 1.75 m length) were arranged at both the en trances of the bridge, and four HD cameras (Hikvision HD-Camera, DS-2 DF8234 4M pixel) were installed along the bridge with overlapping fields of view (FOVs). Th e WIM system equipped each lane with an additioanl camera for recognising vehicle re gistration numbers. (a) Hikvision HD-Camera (b) ME AS Quartz WIM Sensor (c) Installed cameras on the bridge Figure 3. Details of the HD cameras and WIM system For the FBTL monitoring, the WIMs coul d obtain information on each vehicle arriving at the bridge, incl uding its weight, speed, arriva l time, lane number, and registration number. Meanwhile, four cam eras provided FOVs covering the whole bridge deck (shaded in Fig.2) and recorded videos of the vehicle. These videos were used to identify the vehicle position on the bridge by usin g the vehicle detection and tracking algorithms. Then, all the videos obtai ned by the cameras and the text data from WIMs were transferred over an optical fibe r network to the serv er privately owned by the Ningbo Municipal Administration. The cameras and WIM systems have been generating videos and WIM text files respectively. Due to the limitations of the se rver, only the latest 2 months of videos are available to download. Data processing is manually completed on downloaded video and WIMs files. It should be noted that, to fuse the information from the two different sensors, they must be time synchronised. Time synchronisation by GPS time alignment [21] is a promising approach. However, in this study, time sync hronisation was carried out manually, using vehicle registration numbers appearing on the videos and the WIMs text files. 4 Optimised vehicle detection model using YOLO-v4 The first core task of the FBTL monitoring sy stem is to detect vehicles in the video with the aid of computer vision technology, that is, to ma rk the position of vehicles with bounding boxes in each frame of the video seque nces. In fact, the identification of vehicle position has attracted extensive atten tion, and many relevant studies have been carried out in the field of bridge health monitrong. From the in itial application of traditional image processing methods [17,18] to the incorporation of convolutional neural networks [20, 25], the latest developmen ts have been able to accurately identify most vehicles, except for the targets far away from the camera, which present very small pixel clusters in the image. However, for aesthetics and economic considerations, it is common to reduce a number of cameras on th e bridge, expanding the monitoring FOV , resulting in problems with small target detection. This secti on introduces a novel detection model based on YOLO-v4 to addr ess the small target detection problem. 4.1 Image dataset Ve-YJ for model training For training a deep learning model for br idge monitoring, a custom image dataset named Ve-YJ was created, as opposed to using general datasets such as MS COCO and ImageNet. Figure 4. The composition of the Ve-YJ image dataset Ve-YJ consists of a total of 24,348 pict ures, with a resolution of 1,920×1,080, randomly selected from 3 days of mon itoring video taken from 5am to 7pm. The pictures were chosen from different levels of traffic and light conditions, and labeled for three types of vehicles distributed in different positions of the camera FOV (Fig.4). The three classes of vehicles (cars, buses and trucks) were annotated with bounding boxes to record their positions. Inspired by li terature [20], when the vehicle is fully visible in the FOV , its front and rear were also labeled for the correction of transverse vehicle positions on the bridge. Therefore, a total of 9 classes of objects were annotated, i.e, car, car-front, car-rear, bus, bus-front, bus -rear, truck, truck-fr ont and truck-rear. The number and proportion of different classe s in the Ve-YJ are shown in Table 1. Table 1. The number and proportion of different classes in the image dataset Ve-YJ Class car car-front car-rear bus bus-front bus-rear truck truck-front truck-rear Quantity 52706 14937 16764 1245 649 790 547 268 263 Proportion 59.78% 16.94% 19.01% 1.41% 0.74% 0.90% 0.62% 0.30% 0.30% The Ve-YJ dataset was used not only as tr aining and test datasets for the YOLO- v4-based vehicle detection model, but also a dataset for the transverse vehicle position correction detailed in Section 6. 4.2 YOLO-v4 The YOLO (You Only Look Once) [26] ne twork is a single-stage CNN-based object detection algorithm, which directly calculates a classi fication and position coordinates of each target on the input im age. YOLO-v4, which was proposed in 2020, integrates the advantages of various types of target detec tion networks and achieves the best trade off between accuracy and speed (Figure 5). The main backbone network used for training and extracting image features is CSPDarknet53, an open source neural network framework with an improved perfor mance. Path Aggregation Network (PANet) was used as the neck network to fuse extract ed features from diff erent detector levels, instead of FPN in YOLO-v3 [27]. But the head of YOLO-v3 was still retained in YOLO-v4. In the structure of our vehicle detecti on model, the composition and fuctions of main modules are as follows.  The CBL (Figure 5) represents a module co mposed of a convolution layer, a batch normalisation layer and a Leaky-ReLU ac tivation function, which was the most frequently used in the YOLO -v4 for feature extraction.  The CBM module is also used for feature extraction, like the CBL, but with the activation function of MISH instead of Leaky-ReLU.  The SPP, a spatial pyramid pooling layer, which mainly transforms convolution features of different sizes into poole d features with the same length [28].  The CSP refers to the Center and Scale Prediction module that could enhance the learning ability of CNNs by dividing low- level features into two parts and then fusing cross-level features [29]. Moreover, the YOLO-v4 based vehicle det ection model used CIOU [30] as the loss function, which considers the overlap area, ce ntroid distance and aspect ratio of the predicted and ground truth bounding boxes, gr eatly improving the calculation accuracy of the vehicle position in the image. Figure 5. The architecture of the vehicle detection model based on YOLO-v4 The training process of th is detection model involves undertaking the following steps. Step 1: Prepare data. The image dataset Ve-Y J was employed in this study. The labeled 24,348 images were divided into a training set comprising 19,478 images (80%) and a test set with 4,870 images (20%). Step 2: Data augmentation. Before being fed into the network, all images in the training set were scaled down to the re solution of 416×416 and processed, using a variety of data augmentation techniques, including rotation, mirroring, colour and brightness adjustment, and blurring. Step 3: Model training and testing. Refe rring to official weights of the CSPDarknet53 network pretrained from th e COCO dataset, the proposed vehicle detection model was trained and tested on a computer configured with GPU (NVIDA RTX 2070) and CPU (Intel Co re i7-6700@3.4GHz). The training performed 20,000 iterations, and the key pa rameters are set to batch size = 8 and initial learning rate = 0.001. After training, a test is conducted on the test dataset to verify the detection accuracy for different types of targets in comparison to a YOLO-v3 based model [19] as shown in Fig.6. Figure 6. Performance comparision of our method and YOLO-v3 based method In Fig.6, AP@0.5 represents the average precision of the model for an IOU threshold of 0.5. The result showed that the proposed model outperforms the YOLO-v3 based model on detection accuracy for almost all types of targets, except car-front and car-rear. The detection accuracy of cars was much higher than that of buses and trucks. Such test results may be explained by the following two reasons: (1) The front and rear profiles of the cars in the distant area of FOV were not annotated in the dataset Ve-YJ, because the transverse position error is negligible for distant cars. The YOLO-v4 based model is more sensitive to small targets, so it can still detect the unannotated car fronts and rear s in the area far away from the camera, leading to a lower AP for these two classes. (2) In the Ve-YJ, the number of buses and trucks in the image was much lower than that of cars, resulting in lower detecti on accuracy of them. To improve accuracy, it is recommended that the dataset is further expanded or another published vehicle image dataset is integrated, to in crease the samples of buses and trucks. Figure 7. The ability of different methods to detect small targets (left: YOLO-v3 based method[19]; right: YOLO-v4 in this study) To further verify the advantage of th e proposed model, the two methods were applied to a new scene never before used in training, as shown in Fig.7, with the detection results of multiple targets at the fa r end of the FOV (the red dashed rectangle). The result shows that the proposed method demonstrated better performance in detecting small vehicles a nd obtaining more compact bound ing boxes, even if both the two methods show a robust performance wh en applied to the new monitoring scene 5 Vehicle tracking by combining Ka lman filter and apparent feature According to the authors’ previous studies [18,20], Kalman filtering based vehicle tracking achieved a satisfactory result in short-term tracking tasks. However, with a longer tracking time, occlusion between the vehicles in different lanes may occur (see Fig.9), which leads to losing tracked trajetories. A robust tracking algorithm combining Kalman filter and appearance features is disc ussed in this section. 5.1 Image dataset for model training It is necessary to prepare an image da taset to train a deep learning model for extracting vehicle appearance features. Instead of making our own dataset, the dataset named VeRi-776 [23] was used for the model training, with permission from the authors. As shown in Fig.8, VeRi-776 contains over 50,000 images of 776 vehicles, captured by 20 cameras on a one km2 road network. Each vehicle was annotated with a bounding box, colour, type, the IDs of vehicle and camera, helpin g to train the model to distinguish the same car from different views. (a) ( b ) Figure 8. (a) the spatiotemporal information of a vehicle in the surveillance network; (b) vehicle images captured in different viewpoints. (cited from [22]) 5.2 Kalman filter and appearance features In the long-term tracking scenario, it is di fficult to retrieve a lost trajectory when only relying on the prediction of vehicle mo tion information. A new robust tracking system with dual metrics is proposed. The first metric is based on the detect ed vehicle bounding box and predicted ones using the Kalman filter with constant velcit y motion and a linear observation model. In the tracking scenario, the state vector and measurement vector of the Kalman filter is defined as (,,,,,,,)xy hxy h  and (, , ,)xy h respectively, where (, )xy are the bounding box center, is aspect ratio, his height and (, , ,)xy h are their respective velocities in the image coordinates. Using the YOLO-v4 model, it is easy to obtain the measurement vector (, , ,)xy h and the predicted Kalman state (, , ,)ppppxy h in the next frame. Essentially, using the Kalman filter ve hicle tracking is a matching problem between vectors (, , ,)xy h and (, , ,)ppppxy h in each image frame, followed by forming a trajectory for successful matches over multiple frames. The solution of the matching problem and the rules for trajecto ry generation need to be explained. For the matching problem, the Mahala nobis distance was used between the predicted Kalman states and newly arrived measurements on each image frame: 1 1(, ) ( ) ( )T ji i ji dij d p d p  S ( 1 ) where ip denotes the predicted bounding box of the i-th track, Si the covariance matrix of four dimens ional variables, and jd the measurment vector of the j-th bounding box detection. The smaller 1(, )dij means a higher degree of matching. With a threshold value 11(, )Thresholddij d , the newly arrived measurement can be matched with an existing trajectory. Howeve r, in case matching cannot be achieved, the exisiting trajectory stops and new detections may appear. This situation needs to be rectified. For the former, starting from the last successful measurement, the number of frames where track k have been continously ex isited is counted as age ak , which is incremented during the Kalman filter predicti on and set to 0 after succesful matching. The maximum track age is defined as Amax. Tracks that exceed Amax are deleted, because they are considered to have le ft the scene. For the latter s ituation, a detection is assigned to a new trajectory only if there is no matching trajectory for consecutive bk frames ( bk = 5 in this study). The second metric is the similarity of ve hicle appearance features, which was also integrated to the matching problem in this study. For the j-th bounding box detection, an appearance feature vector jf is computed and normalised 1jf . Furthermore, a feature gallery1 {}kNi kk i f F of the last 100kN frames is kept for each track k. Then, the second metric to measure the similarity between the i-th track and j-th detection is defined as: 2(, ) m i n { 1 | }Ti i j kk k di j f f f   F ( 2 ) In practice, there are many methods that can be selected for the extraction of image features in the bounding box, including the traditional methods such as SIFT [31] and HOG [32] and deep learning methods. Herein , CNN architecture th at has used for person re-identification[33-34] was chosen. Th is CNN is relatively shallow, to allow fast training and inference, as shown in Table 2. All the input images are rescaled to 128×64 and presented in RGB colour space. Af ter a series of convol ution operations, a 128-dimensional feature v ector is extracted by Dense 10 and then projected onto the unit hypersphere by the l2 normalisation to output the appearance feature vector jf . The image datasets VeRi-776 and Ve-YJ we re employed in the training process of the CNN for further performance improvement. Table 2. Overview of the CNN architectur e for vehicle appearance feature extraction Name Patch Size/Stride Output Size Conv 1 3×3/1 32×128×64 Conv 2 3×3/1 32×128×64 Max Pool 3 3×3/2 32×64×32 Residual 4 3×3/1 32×64×32 Residual 5 3×3/1 32×64×32 Residual 6 3×3/2 64×32×16 Residual 7 3×3/1 64×32×16 Residual 8 3×3/2 128×16×8 Residual 9 3×3/1 128×16×8 Dense 10 128 l2 normalisation 128 Figure 9 shows the typical tr acking process of an occluded vehicle based on the metric d1 only (Fig.9 left) and a combination of the two metrics (Fig.9 right), where 3 different image frames are used for illustration. In frame 621, it was observed that the grey car (No. 8) was about to change lanes and overtake the white car (No. 6). Then, in the 627 th frame, Car 8 changed lanes and se verely occluded Car 6, making Car 6 undetectable. In this case, our goal is to r ecapture Car 6 after occlusion. After losing track for dozens of frames, Car 6 was detected again at the 640 th frame. As shown in Fig.9, the photos at frame 640 pr esented the re-identification results of Car 6 using the two methods. Obviously, the only d1-based method renumbers the white car as 11, losing the trajectory of Car 6. However, even if the trajectory of Car 6 was lost for more than ten frames, the proposed tracking sche me recovered its iden tity and achieved continuous tracking. Figure 9. Tracking results when encountering occlusion (left: only d 1-based method, right: the proposed method) It should be noted that the Kalman filte r method may continue to predict vehicle position without measurement. In theory, by adjusting the maximum track age Amax, the predicted data can be retained and used to reidentify the lost trajectory. To investigate this possibility, the case in Fig.9 was considered. Figure 10 showed the photo at the 640 th frame when Car 6 was detected. The de tected bounding box of Car 6, the predicted bounding boxes of Car 6 and Car 8, and the Mahalanobis distance d1 were marked in Fig.10. It was seen that, after a dozen frames of trajectory loss, the predicted result of Car 6 deviated so much that the d1 from the detected bounding box of Car 6 to the predicted bounding box of Car 8 was smaller than that of its own predicted bounding box. This demonstrated th at using the metric d1 alone made it difficult to retrieve a lost trajectory, while the proposed dual-m etric tracking method effectively overcame the occlusion problem. Figure 10. Detected and predicted bounding boxes of Car 6 and Car 8 at frame 640 6 Fast vehicle transverse position correction using regression By training the model based on th e YOLO-v4, obtaining the bounding boxes of the vehicle and its front and rear parts was achievable. Based on the bounding boxes of the detected vehicles, vehicle fronts and rears, an accurate position correction also proved accessible in our previ ous study [20]. An efficient correction method is proposed using whole vehicle detection and a pre-determined mathmatical relationship, rather than time-consuming dual bounding boxes detection. Figure 11 presents part of an image taken by Camera 3 on the Yongjiang Bridge. The red and purple dashed lines marked th e sides of the bounding boxes, and the central axis of the front and rear profiles were indicated by yellow dashed lines, whose corresponding image coordinate xcenter is the vehicle position to calculate. Based on the image coordinates presented in Fig.11, the correction factor R is defined as: 32 31j j j j jxxRxx ( 3 ) Figure 11. Key points of correction factor calculation in the image Assuming that the camera shooting direction is parallel to the direction of the traffic flow, the relationship between the image coordinate x and the bridge deck coordinates ( X, Y, Z ) was derived based on the projective geometry [20]. () sin ( ) cos ( )x yzfX DxYD ZD  ( 4 ) where f is the camera focal length, and α is the shooting angle of the camera ( α=0 when shooting vertically upward). Dx, Dy and Dz are the distances from the origin of the bridge deck coordinate system to the camera in the three directions, respectively. Since all the key points in Fig.11 (marked as triangles) are on the pavement, Z is set to 0 here. The image coordinates in Eq.(3) can be expressed as: 1 1 1 1 2 1 1 3 1() sin ( ) cos () sin ( ) cos () sin ( ) cosj j x j yz j jx j j yz j jx j j j yzfX DxYD D fX W DxYD D fX W DxYL D D            ( 5 ) where Wj and Lj refer the width and length of the j-th vehicle. By substituting Eq.(5) into Eq.(3) 1 11() s i n cos ( )sinj jj j jj zj j jLW XRDW L X WY   ( 6 ) According to the transformation procedure introduced in [18], all the image coordinates ( x, y) can be converted to br idge deck coordinates ( X, Y, Z ) as follows 11 12 13 14 21 22 23 24 31 32 33 34 1 11jj j jj cj j jjX X x mmmmYY Zy m m m m Z Zmmmm                      M ( 7 ) In this equation, Zcj denotes the distance from the spatial point j (point 1 in Fig.11) to the camera optical centre, and M is the transformation matrix. Eq.(6) can be rewriten as 11 1 12 1 13 11 1 12 1 13 21 22 23() ( ) c o tjj cj cj cj j j j jj cj cj z jZmx Zmy Zm WRWZm x m y m Zm m m DL            ( 8 ) where ijm is the element of the inverse matrix of M. It is seen that the variables except x, y, W and L in the above formul a are constants for a single camera. The vehicle width W is often small enough to be neglected relative to the other terms of the numerator. It is a reasonable assumption that the change in the ratio of the width to the length of a vehicle class is very sma ll. For each class of vehicle, the correction factor R can be approximately expressed as a polynomial in the following form. 11 11 1 21 21 2jj j jjAxB yCRAxB xC ( 9 ) The parameters ( A1, A2, B1, B2, C1, C2) can be determined by performing a regression on the coordinates data in the dataset Ve-YJ. For the j-th vehicle, the final coordinates used to calculate the transverse position are 1 (, )j j centerx y, in which j centerx is obtained by 31 13() () 22jj jj j j centerRxx xxx  ( 1 0 ) With the correction factor R, it becomes possibl e to only detect vehicles without the need to detect their front or rear, greatly saving the calculation time. A test of a video of the scene in Fig.9 showed that the cal culation time can be reduced by about 40% from a 0.05 sec/frame to a 0.029 sec/frame. The relative errors of the factor R were investigated for all vehicles in the dataset Ve-YJ, and shown in Table 3 according to the camera number and vehicle type. Table 3. The relative errors of R for different vehicles and FOVs Types Car& Front Car& Rear Bus& Front Bus& Rear Truck& Front Truck& Rear No. of FOVs Error max 12.59% 12.56% 7.89% 11.51% 11.49% 12.93% Camera 1 Error average 2.53% 2.59% 2.08% 2.10% 2.82% 2.79% Pr (Error>10% ) 0.28% 0.50% 0.00% 0.76% 1.10% 1.64% Error max 14.11% 15..17% 10.44% 9.88% 9.53% 12.06% Camera 2 Error average 2.72% 2.73% 2.02% 2.08% 3.15% 3.22% Pr (Error>10% ) 0.84% 0.68% 0.80% 0.00% 0.00% 3.03% Error max 13.44% 13.99% 10.94% 17.00% 10.34% 11.24% Camera 3 Error average 2.09% 3.86% 2.34% 2.20% 3.36% 3.25% Pr (Error>10% ) 0.10% 1.21% 0.39% 0.87% 0.71% 1.20% Error max 10.76% 13.67% 8.51% 10.66% 7.13% 10.19% Camera 4 Error average 1.65% 2.04% 2.59% 2.32% 3.02% 3.00% Pr (Error>10% ) 0.08% 0.31% 0.00% 0.60% 0.00% 1.33% Note:. ‘Vehicle’&Front represents the oncoming vehicle with a visible front, and ‘Vehicle’&Rear the departing vehicle with a visible rear. Pr (Error>10%) indicates the probability that the relative error is greater than 10%. It can be seen from the results that th e maximum relative error of the correction factor is 17%, the average re lative error is about 3%, and th e probability of the relative error exceeding 10% is also very low (<1% in most cases). 7 Coordinate transformation from image to bridge coordinates Identification of the external and inte rnal camera parameters are required for transformation from image coordinates to bridge coordinates. External camera parameters refer to the transformation matr ix between the camera image coordinates and bridge deck coordinates. In general, the transformation matrix is estimated using the coordinate pairs of the image c oordinates and corresponding bridge deck coordinates of reference points arranged on the bridge deck, which has been introduced in detail in [18]. In this st udy, a field test was conducte d on the Yongjiang Bridge with a total of 49 marked PVC boa rds arranged along the bri dge (Fig.12). The precise coordinates of the reference points on the bridge deck were all measured by a total station, while their image coordinates were manually read from the images. Figure 12. Field test for the calculation of external parameters The camera internal parameters were es timated using a checkerboard and Zhang method [23] to correct the lens distortion (Fig.13). Figure 13. Field test for the calib ration of intern al parameters In order to verify the accuracy of th e coordinates’ transformation matrices identified from the field test, three refere nce points in each FOV were excluded from the calculation of transfor mation matrix (eleven yellow markers in Fig.12), and the errors were calculated as shown in Table 4. It showed the errors of the eleven yellow reference points and the average and maximu m errors of all reference points, where Xerror indicated the errors in the transverse bridge direction and Yerror in the longitudinal bridge direction. Table 4. Transformation errors of reference points in the field test (unit: m) No. of reference points FOVs X yellow error Y yellow error X average error Y average error X maximum error Y maximum error 8 Camera 1 0.199 0.811 0.047 0.225 0.120 0.458 23 0.062 0.509 36 0.000 0.128 7 Camera 2 0.015 0.166 0.055 0.360 0.183 0.943 21 0.018 0.103 34 0.144 1.153 11 Camera 3 0.332 0.108 0.059 0.473 0.206 1.350 30 0.032 0.243 49 0.006 0.043 11 Camera 4 0.059 0.425 0.051 0.266 0.216 0.657 25 0.077 0.288 42 0.196 0.610 From the results, the average value of th e coordinate transformation error of the reference points was about 0.05m in the tran sverse direction and le ss than 0.5m in the longitudinal direction. As for the maximum error, it was 0.216m in transverse direction while 1.35m in longitudinal direction. 7.2 Full-bridge traffic load dist ribution estimation result Twenty four hours of videos and WIMs te xt data collected on November 16, 2020 and four-minute continuous segments from 4:49 p.m to 4:53 p.m were selected for a demonstration of the proposed long-term FBTL monitoring system. Figure 14 shows the visualisation of analysis results for two calculation frames (mor e results can be seen in the attached videos). (a) The 2302 th calculation frame (b) The 4420 th calculation frame Figure.14. The full-bridge traffic load distribution on Yongjiang Bridge In Fig.14, the bridge deck of Yongjiang Bridge is represented by the X-Y plane, vehicle centroid by the centre of the bottom su rface of the square column, and the load value (vehicle weight) by column height. It is worth noting that, in most cases, a complete traffic load distribution on the br idge was estimated as shown in Fig.14(a). However, in a few cases, the WIM system may have missed detection on the vehicle weights, resulting in the absence of load value as shown in Fig.14(b). The vehicle indicated by the blue bar had no weight attached while all other vehicles in the same lane did. In the four minut es, a total of 36 vehicles pa ssed through lanes 1, 2 and 3, while 4 vehicles were missed by the WIM sy stem. The WIMs in lane 4 failed during the test period and were not included in the statistics. To summarise from the cases discussed above, even if the current WIM technology has been well developed, there may still be detection failures or missed detections because of the combined influence of in stallation quality, operating environment and traffic conditions. A special design in the info rmation fusion has been made, so that the proposed system can accurately match the dete cted vehicle to the corresponding weight information, but also automatically identif y and label the missing WIM data based on the video analysis resu lts and time clues. 8 Conclusion This study proposed a long-term full-bridge traffic load monitoring framework and demonstrated its first applic ation on a long-span cable-stayed bridge. Three improved methods were proposed for accurate vehicle de tection, robust vehicl e tracking, and fast vehicle positioning error correc tion as summarised below.  Novel methods for vehicle detection a nd tracking based on deep CNNs were developed on the image dataset (Ve-YJ) dedi cated to bridge traffic scenes, greatly improving the ability of the existing FB TL framework to cope with challenging long-term monitoring tasks.  The proposed correction method of vehicle transverse location achieved a faster computation speed and is suited to cases where a large number of targets need to be processed.  The relevant algorithms of the proposed FBTL monitoring system have been demonstrated on a cable-stayed bridge, and the system is also applicable to any other forms of bridges (suspe nsion bridges, variable widt h bridges and even curved bridges). Combined with the monitoring of bridge structural response, the proposed system is expected to be used as a tool in some important issues, such as load analysis, model updating and damage identification. With th e rapid development of deep learning and sensing technologies, the propos ed framework could be further improved in the future. Acknowledgement This work is supported by the Na tional key R&D Program of China (2017YFF0205605); the National Nature Scie nce Foundation of China (Grant No. 5187849); Shanghai Urban Construction Design Research Institute Project ‘Bridge Safe Operation Big Data Acquisition Technology and Structure Monitoring System Research’; the Ministry of Transport Construction Science and Technology Project ‘Medium-Small Span Bridge Structure Network Level Safety Monitoring and Evaluation’; and the CSC (China Schol arship Council) Sc holarship (202006260246).