# Anomaly Detection of Highway Vehicle Trajectory under the Internet of Things Converged with 5G Technology

Authors: Ketao Deng
Journal: Complexity
 (2021)

---

## Abstract

The gradual increase in the density of highway vehicles and traffic flow makes the abnormal driving state of vehicles an indispensable tool for assisting traffic dispatch. Intelligent transportation systems can detect and track vehicles in real time, acquire characteristics such as vehicle traffic, vehicle speed, vehicle flow density, and vehicle trajectory, and further perform advanced tasks such as vehicle trajectory. The detection of abnormal vehicle trajectory is an important content of vehicle trajectory understanding. And the development of the Internet of Things (IoT) and 5G technology has led to a continuous increase in the rate of data information circulation. The “Internet of Vehicles” generated based on the practice of 5G communication technology constitutes a vehicle abnormal trajectory detection system, which has very high feasibility and safety and stability. Therefore, this research is aimed at the needs of preventing major accidents and forensic analysis during highway vehicles. Based on the integration of the Internet of Things 5G communication technology, a trajectorial anomaly detection of highway vehicle trajectory based on the integration of the Internet of Things 5G is proposed. By accurately sensing unsafe events at the perception layer, network layer, and application layer, the vehicle driving trajectory state is divided into several simple semantic representations. The semantic representation is analyzed, and then the moving target detection and moving target tracking algorithms needed to extract the vehicle trajectory are introduced. Through video detection and tracking of moving vehicle targets, the driving trajectory of the vehicle is obtained, and the movement characteristics of the vehicle in each frame of image are extracted. According to the relationship between the trajectory of the vehicle and the lane line, the vehicle trajectory analysis is realized, and then it is judged whether the vehicle has abnormal trajectory. Compared with the traditional method of manually detecting the driving condition of the vehicle, the abnormal trajectory detection of the vehicle based on the integration of the Internet of Things and 5G can quickly detect the abnormal trajectory of the vehicle in the traffic monitoring video.


---

## Body

## 1. Introduction

With the increase of road vehicles, traffic has become more and more congested, and the rate of traffic accidents has become higher and higher, especially on highways. Once the phenomenon of abnormal driving such as illegal stop and retrograde occurs, it is often accompanied by major traffic accidents, causing serious consequences. The traditional judgment of abnormal vehicle driving requires manual viewing of surveillance videos to monitor and manage traffic conditions [1]. However, the efficiency of this method is very low, the work intensity of the staff is also very high, and there are problems that are missed and cannot be found in time, which cannot meet the needs of current operation management. At the same time, current vehicle abnormal trajectory detection still faces problems such as lack of data, inaccuracy of abnormal definition, occlusion, and poor real-time performance in practical applications.The development of the Internet of Things (IoT) and the Internet has led to a continuous increase in the rate of data information circulation. As the fifth-generation cellular mobile communication technology, 5G has large-capacity wireless network technology and high-speed wireless transmission technology that surpass 4G. This can be integrated with the large-scale Internet of Things that blows out network requests and network traffic to realize the real interconnection of everything. 5G communication technology can take advantage of its own advantages such as low cost, large capacity, and low energy consumption to flexibly deploy and operate logistics networks, improve the utilization of network space in the process of highway vehicle management, control vehicle driving, and solve the problems faced by vehicles during driving [2, 3]. Therefore, an automatic detection system for abnormal events of highway vehicles based on the integration of the Internet of Things and 5G technology came into being. Through real-time processing and analysis of traffic monitoring video data, it can automatically detect and identify abnormal trajectory of vehicles on the road, such as speeding, sudden braking, illegal steering, unauthorized lane changes, running red lights, and vehicles’ going backwards. The abnormal driving of the vehicle is likely to cause traffic accidents. In the early stage of abnormal vehicle driving, rapid detection and timely warning and trajectory restraint can significantly reduce the incidence of traffic accidents [4]. However, the monitoring of the remote abnormal driving trajectory of existing vehicles mainly returns data through the Internet of Things card, which is limited by the transmission speed and can only transmit small-scale data. Under the existing network transmission conditions, it is difficult to support a large number of vehicles to send back large-scale data (such as high-definition video) concurrently in real time [4]. How to achieve automatic detection of traffic incidents efficiently, accurately, and quickly is still a major problem facing the current highway traffic field.Therefore, this research was based on 5G communication technology and the Internet of Things fusion system technology, taking the driving video of highway vehicles as the research object. Starting research from several key technologies such as moving target detection, tracking, and abnormal trajectory description, an automatic detection algorithm was designed for abnormal driving on expressways. Through the Internet of Things converged 5G communication technology, problems such as the serious lack of transportation safety and emergency response mechanisms in the information age were solved. The application of the system uses networked and informational means to carry out automated and intelligent safety monitoring and strategy formulation, which is expected to fundamentally reduce the safety risks of the Internet of Vehicles information system, reduce the occurrence of traffic safety accidents, and reduce casualties and economic losses.


## 2. The Application of IoT and 5G Technology in the Driving Trajectory of Highway Vehicles

Detecting the driving of highway vehicles has been relatively mature for the application of the Internet of Things technology. However, with the abnormal expansion of the trajectory of highway vehicles, the existing communication technology has limitations in terms of safety, real-time performance, and transmission rate. As the latest level of current communication technology capabilities, 5G’s interconnection of everything has greatly promoted the development of the Internet of Things [5].


### 2.1. The Role of IoT and 5G Technology on Highway Traffic

In the context of the Internet of Things, the practical advantages of 5G communication technology are mainly reflected in the fact that this technology can increase data traffic, expand equipment, and make communication more reliable. 5G technology allows users to directly handle some network services on the network platform, saving users time and ensuring the reliability of communication services. Therefore, in the context of the Internet of Things, 5G communication technology can realize “things and things” communication services. And according to the scenarios required by the Internet of Things, the communication protocol is customized to enable the Internet of Things to have the function of the “Internet of Things.” The integration of the Internet of Things and 5G communication technology realizes the communication interconnection between vehicles and vehicles, vehicles and people, vehicles and road infrastructure, and vehicles and network service platforms (as shown in Figure1) [6]. “Vehicle to vehicle” can monitor the information between vehicles in the system in real time and realize the functions of vehicle-assisted driving, emergency collision warning, vehicle lane change and steering assist. “Vehicle and person” establishes the interconnection of information between vehicles and pedestrians, which can avoid collisions between vehicles and pedestrians and ensure the safety of pedestrians. “Vehicle and network” realizes the network interconnection between the vehicle and the network platform in the vehicle network system, and the network platform can provide navigation information and road traffic information for the vehicle. “Vehicles and road” provide basic road information for vehicles, such as road traffic conditions and speed limit information, so that violations caused by speeding vehicles can be effectively avoided.Figure 1
The Internet of Things converged 5G communication technology to get off the communication interconnection between “vehicles and vehicles,” “vehicles and person,” “vehicles and road,” and “vehicles and network service platforms.”


### 2.2. Research Status of IoT Fusion 5G in Vehicle Trajectory Detection

The highway vehicle trajectory detection system based on the Internet of Things and 5G communication technology can effectively divide the management between systems, accurately locate the physical location, and form an effective management plan so as to improve the safety of highway vehicles. Video surveillance system is one of the key elements to ensure the safety of highway vehicles. Using 5G’s ultra-high bandwidth characteristics to transmit video can realize remote monitoring of the vehicle and upload the video to the cloud for backup and storage. More and more researchers pay attention to the research of vehicle driving state detection. TheDouble Exponential Smoothing algorithm was mentioned in the literature [7], which used the actual and predicted values of traffic parameters to design and track vehicle driving signals. The Bayesian algorithm developed by Karagiannis et al. used the difference in occupancy between two adjacent detectors to determine whether traffic congestion occurs [8]. The dynamic model algorithm developed by Li et al. used the relationship between “flow-density” and “speed-density” of traffic flow to determine whether a traffic incident occurs [9]. Misbahuddin developed a low-pass filtering algorithm using a moving average method, which filters out the high-frequency and noise components in the measured traffic parameters and only retains low-frequency data to judge traffic congestion [10]. Literature [11] proposed an automatic detection method for traffic incidents based on a video-based vehicle trajectory model and established a mathematical model to determine the type of traffic incidents that occurred.


### 2.3. Internet of Things Fusion 5G Technology Used in Highway Vehicle Trajectory Detection

The most widely used sensor devices for the perception layer of the Internet of Things are barcodes and cameras. Its key technologies include radio frequency identification, barcodes, sensors, and smart embedded devices. The network layer of the Internet of Things generally uses wireless networks, the Internet, and wired networks for data transmission. The network layer mainly uses the integration of various telecommunication networks (such as 4G/5G) and the Internet to transmit data safely, completely, and quickly. The 5G technology applied to the network layer; the main technologies include ZigBee, RFID, NFC, Wi-Fi, Bluetooth, Global Positioning System (GPS) [12]. For highway vehicle monitoring, the application layer of the Internet of Things can collect vehicle information on the road through sensors such as coils and cameras. The data is transmitted to the management platform through the network, and statistics and management of information such as traffic flow, vehicle speed, and vehicle occupation ratio on the road are performed. In this research, the GPS technology and RFID technology are used. In highway traffic, GPS technology can not only provide the driver with vehicle positioning information, but also automatically select the best driving route for the driver as well as provide road traffic information and other query information. The GPS-based vehicle highway monitoring system consists of three parts: a vehicle-mounted terminal, a monitoring center, and a wireless communication network, as shown in Figure 2 [13].Figure 2
Vehicle highway monitoring system based on GPS technology.The RFID technology can automatically identify vehicle information, detect the driving route and driving status of the vehicle, collect road traffic information, provide road traffic flow data, and realize automatic charging. When the vehicle-mounted electronic tag passes the RFID reader set up on the side of the road, the RFID system will automatically record the elapsed time of the vehicle and read the basic information of the vehicle and pass it to the monitoring center. Once the monitoring center finds abnormal trajectory, it can automatically call the police and remind the staff to pay attention to the whereabouts of the vehicle and check it.


### 2.4. Research on the Highway Vehicle Driving State Detection System Model under the Framework of Internet of Things and 5G

The Internet of Things integrates 5G technology, and their interactive nodes are on-board computers. The in-vehicle system is connected through the 5G network to process data. The integration of 5G technology with the Internet of Things is the most important part of the field of intelligent transportation. Like the Internet of Things, the structure of the Internet of Things converged with 5G technology also has three levels: front-end perception layer, network transmission layer, and application layer (Figure3) [14]. The perception layer is responsible for the collection of vehicle information, and all data in the Internet of Things originates from this. With the help of 5G communication technology, it is possible to obtain the ultra-high bandwidth characteristics of the transmission video during the driving of highway vehicles. The second is the network transmission layer, which is responsible for the transmission of information. The data in the integration of the Internet of Things 5G technology can be obtained anytime and anywhere. The third is the application layer, which realizes the recognition and perception between objects and people [15].Figure 3
Architecture diagram of IoT fusion 5G technology.This research was based on the integration of 5G technology and architecture diagram of the Internet of Things, combining vehicle speed detection system and vehicle abnormal trajectory detection to form a highway trajectory driving state detection system, proposing the highway trajectory driving state detection system in this paper. The video information was transmitted through 5G ultra-high bandwidth characteristics; the video data and image data obtained by the Internet of Things were processed to realize the function of vehicle speed detection and vehicle trajectory analysis. This system was conducive to promoting the management and control of intelligent traffic on expressways and reducing the occurrence of traffic accidents. This part first introduced the semantic representation and feature analysis of vehicle traffic trajectory and then introduced the moving target detection and moving target tracking algorithms needed to extract the vehicle trajectory. Finally, the vehicle trajectory semantic analysis algorithm designed and its implementation steps were introduced, and the results of the experimental data were summarized and analyzed.


### 2.5. The Framework and Process of the Internet of Things and 5G-Integrated Highway Vehicle Driving State Detection System

If we want to realize the detection of abnormal driving vehicles on the highway, you need a complete highway trajectory driving state detection system. The system studies the perception layer, network layer, and application layer of the Internet of Things under 5G technology to build an integrated highway vehicle abnormal driving detection scheme. Based on the architecture diagram of the Internet of Things fusion 5G technology in Figure3, this study constructed the framework of the highway vehicle driving state detection system shown in Figure 4.Figure 4
The framework of the highway vehicle driving state detection system based on the integration of the Internet of Things and 5G technology.The basic goal of the framework is to decouple the real-time driving state of the vehicle from the transmission and storage of large-scale data so as to achieve the dual goals of real-time event detection and large-scale data transmission [16]. According to the dynamic trajectory of the vehicle and the available 5G communication base stations and WiFi hotspots provided by the fixed infrastructure along the road [17], we optimized and dynamically adjusted the time and place of unloading the driving data of different vehicles so as to supplement the limited transmission bandwidth provided by traditional mobile network base stations. After the vehicles are networked on a large scale, it is necessary to store the detected data according to the vehicle trajectory and provide fast retrieval services of multiple attributes. Based on this, the software module flowchart of the highway vehicle driving state detection system designed in this research is shown in Figure 5.Figure 5
The software module flowchart of the highway vehicle driving state detection system integrated with the Internet of Things 5G.


### 2.6. Research on Algorithms of Vehicle Motion Track Detection on Expressway

This research is based on the integration of the Internet of Things and 5G technology and used the video sequences collected during the driving of highway vehicles as the research object. Vehicle abnormal trajectory detection based on highway vehicle driving video generally includes three parts: moving vehicle detection, moving vehicle tracking, and abnormal trajectory recognition. Among them, the detection of moving vehicles is the basis and primary condition for the subsequent tracking of moving vehicles and the recognition of abnormal trajectories. Moving vehicle detection is the process of extracting moving vehicles by analyzing continuous images. It belongs to a kind of moving target detection, so before moving vehicle detection, this research first analyzes and researches moving target detection. At present, the detection algorithms for moving targets mainly include optical flow method, frame difference method, and background difference method [18].


#### 2.6.1. Frame Difference Method

The frame difference method is an algorithm that subtracts two or more adjacent frames in a video frame image sequence to obtain the foreground area of a moving target. When there is a moving target in the video frame image, there will be obvious pixel changes in this area, but there is no pixel change phenomenon in stationary objects. Therefore, the pixels between frames are subtracted, and the pixel difference is compared with a preset threshold to determine whether there are moving objects in the image sequence, as shown in the following equation:(1)Da,b=1fia,b−fi−1a,b≥tm,0fia,b−fi−1a,b≥tm.Among them,fia,b is the image pixel of the current frame, fi−1a,b is the image pixel of the previous frame, tm is the threshold, and Da,b is the binary image pixel after the difference. Making difference between two frames. If the difference is greater than the threshold, it is the foreground; otherwise, it is the background.The method is simple and easy to implement, has low complexity, and is not sensitive to changes in the external environment such as light and weather. However, there are requirements for the speed of the vehicle. Too fast or too slow will affect the detection results of the vehicle, and it is prone to “holes” and “ghosts.”


#### 2.6.2. Optical Flow Method

Optical flow field is a kind of performance movement of image gray mode. It can be calculated mainly by four methods based on gradient, matching, capability, and phase. In a frame of video image, suppose the gray value of pixela,b at time t is Fa,b,t. At time t+Δt, the pixel moves to the next new position a+Δa,b+Δb, and the corresponding gray value is Fa+Δa,b+Δb,t+Δt. Assuming dFa,b,t/dt=0 according to the consistency of the image, the brightness of the image at time t and t+Δt remains unchanged; namely, (2)Fa,b,t=Fa+Δa,b+Δb,t+Δt.Assuming that the two components of the optical flow vector of the pixel along thea, b directions are represented by u, v, respectively, and there is u=da/dt,v=db/dt, the Taylor series expansion of equation (2) is obtained:(3)Fa,b,t=Fa+Δa,b+Δb,t+Δt≈Fa,b,t+∂F∂ada+∂F∂bdb+∂F∂tdt.WhenΔt⟶0, the basic equation of optical flow can be obtained:(4)∂F∂adadt+∂F∂bdbdt+∂F∂t=Fau+Fbv+Ft=0.Among them,Fa,Fb,Ft represent the change of the pixel gray value with a, b, t, respectively.This method can accurately detect and identify the location of the moving target without knowing the scene information. At the same time, the camera is still applicable when in motion. However, this method requires a lot of calculation and time. Since changing light will be mistakenly recognized as optical flow, this method is sensitive to light and will affect the recognition effect.


#### 2.6.3. Background Difference Method

The background difference method uses the difference between the current frame image and the background image and performs filtering and morphological processing on the result to obtain the foreground of the moving target, as shown in the following equations:(5)Da,b=fia,b−Bi−1a,b,(6)Fa,b=1Da,b≥tm,0Da,b≤tm.Among them,fia,b is the pixel value of the current image, Bi−1a,b is the pixel value of the background image, and Fa,b is the binarized foreground pixel value of the moving target. Through comparison, it can be seen that the background difference method is simple, and the calculation speed is fast, and a complete and accurate description of the moving target area can be obtained. It can detect both slow-moving vehicles and temporarily parked vehicles, and it meets the detection requirements of abnormal vehicle driving conditions in this paper. This method is more dependent on the background. The better the purity of the extracted background, the better the foreground effect of the detected moving target. Based on the calculation principle of the Internet of Things and 5G communication technology, this research proposes an adaptive threshold background difference method.The background difference method with adaptive threshold proposed in this study also meets the principle of the background difference method, except that the adaptive threshold algorithm is used to determine the thresholdH. This threshold will change with changes in the foreground and background, so as to optimize the extraction of moving objects. To determine the adaptive threshold H, it is necessary to calculate the average value Ddiff and standard deviation Δdiff of the interframe difference of each pixel, as follows [19]:LetIna,b represent the gray value of the nth frame image at pixel point a,b and inter represent the interval between two frames, which is set to 5 in this study. Let Fna,b be as follows:(7)Fna,b=Ina,b−Iintera,ba,b,Ddiffa,b=1M∑interMFa,b,Δdiffa,b=1M∑interM∑Fna,b−Ddiffa,b2.In order to ensure the accuracy ofDdiff and Δdiff, M is usually large enough. In this study, M is set to 35+inter. After obtaining Ddiff and Δdiff, the threshold H can be determined as follows: (8)H=Ddiff+βΔdiff.After the detection of each frame of image, the background model must be updated. For all pixelsa,b, let Da,b,Ddiffa,b,Δdiffa,b be, respectively, D′a,b,Ddiff′a,b,Δdiff′a,b after update, which are shown in the following equations:(9)D′a,b=I−αDa,b+αIa,b,Ddiff′a,b=I−αDdiffa,b+αFa,b,Δdiff′a,b=I−αDdiffa,b+αFa,b−D′a,b.


### 2.7. Stable Tracking of the Driving State of Highway Vehicles with the Integration of the Internet of Things and 5G

The process of vehicle tracking is actually the process of feature matching, so it is necessary to select the appropriate feature and the optimal matching strategy. There are many characteristics of vehicles, including regional blocks, gradients, textures, depth, and optical flow [20]. The matching methods for selecting different features are also different, and their time and space complexity are also different. Sometimes, in order to increase the accuracy of matching, it is difficult to realize it, and it is necessary to continuously track the vehicle. The perception layer under the integration of the Internet of Things 5G technology only needs to use a high-definition camera to collect the data required by the system. Therefore, the highway vehicle abnormal trajectory detection method based on high-quality video can not only obtain real-time traffic status information more intuitively and comprehensively, but also has higher reliability [21]. This article started from the perspective of video, through real-time analysis of vehicle video, detected and tracked related vehicles, and detected vehicles with abnormal trajectories in time. As shown in Figure 6, a video-based vehicle abnormal trajectory detection system usually includes a camera, an optical transceiver, a communication device, a video detection processing unit, and a display unit. The camera is responsible for collecting video in real time; the video detection processing unit is responsible for the processing of the vehicle video, extracting useful information of the vehicle from it, and identifying the vehicle with abnormal trajectory. The data computer is used to control the system and store information about vehicles that have abnormal trajectories.Figure 6
High-speed vehicle abnormal driving detection system structure based on high-definition video under the background of the integration of the Internet of Things and 5G.The camera is generally set directly above or on the side of a road in a city, and the height is generally between 5 and 10 meters. The schematic diagram and example of the spatial position of the camera and the road vehicle are shown in Figure7 [22].Figure 7
Schematic diagram of the spatial position of the camera and road vehicles (a) and example (b).
(a)(b)At the same time, in order to easily obtain the actual displacement of the vehicle movement, it is necessary to understand the unified standards of the lane lines on the roads in China. The highway is divided into multiple lanes by dashed white lines. Among them, the length and width of the white solid line in the dashed white line and the longitudinal interval between the two white performances are fixed, and the width of the lane divided by the lane line is also fixed, with a value of 375 cm [23]. The standard maps of the Chinese highway lane lines are shown in Figure 8(a). Because of the vehicle speed detection in this study, all the data obtained comes from the camera. Therefore, the standard map of the Chinese highway lane lines will be seen through and become a trapezoid-like shape (Figure 8(b)). The vehicle coordinates obtained by processing the video data are pixel coordinates and need to be converted into actual physical coordinates to obtain the actual displacement of the vehicle movement. The semantic analysis of vehicle trajectory in this article combines the relationship between vehicle trajectory and lane lines to analyze and derive the semantics of vehicle trajectory. Therefore, firstly, it is necessary to label the lane segments separately for the background images in the experimental data single-lane traffic video and multilane traffic video. The labeling result is shown in Figure 8(d), where the red line segment is the lane line labeling.Figure 8
The standard map example (a) and schematic diagram (b) of the Chinese expressway lane lines, the real picture (c), and the labeling result map of the lane lines (d).
(a)(b)(c)(d)


### 2.8. The Abnormal Recognition of the Driving State of Highway Vehicles Based on the Integration of the Internet of Things and 5G

Recognition of abnormal trajectory of vehicles on highways is a complex process, involving trajectory learning and trajectory discrimination, which requires high recognition stability and robustness [24]. Therefore, this article mainly focuses on abnormal driving trajectories such as U-turn, continuous lane change, obstacle avoidance, and emergency braking in the highway environment. These trajectories often indicate that there may be abnormal traffic conditions, or they may cause accidents themselves.


### 2.9. Semantic Representation of Abnormal Vehicle Trajectory

The vehicle trajectory obtained from the highway vehicle abnormality detection system can be used to analyze and identify vehicle driving trajectory [25]. Through a lot of observation and analysis, this article divides the vehicle driving trajectory into several simple semantic representations, such as normal driving, retrograde, stop driving, U-turn and retrograde, lane change, overtaking, etc. The semantic representation of vehicle trajectory is shown in Figure 9, among them, (a) is the trajectory of normal driving, (b) is the trajectory of reverse, (c) is the trajectory of parking, (d) is the trajectory of changing lanes, and (e) is the trajectory of overtaking. After the vehicle’s trajectory is obtained through the moving object detection and moving object tracking algorithm, in order to realize the semantic analysis of the vehicle driving, it is first necessary to extract the movement characteristics of the vehicle trajectory. Only by selecting appropriate motion characteristics can the vehicle trajectory be accurately described. Commonly used movement characteristics are target position, movement speed, and movement direction.Figure 9
Schematic diagram of the semantic expression of the vehicle trajectory used in this study. (a) Trajectory of normal driving, (b) trajectory of reverse, (c) trajectory of parking, (d) trajectory of changing lanes, and (e) trajectory of overtaking.
(a)(b)(c)(d)(e)


### 2.10. Motion Feature Extraction of Vehicle Trajectory

To realize the semantic analysis of the vehicle trajectory, it is necessary to extract the motion characteristics of the vehicle trajectory. Commonly used motion characteristics are target position, motion speed, and motion direction. The information of the target position of the vehicle is an important indicator for semantic analysis and understanding of vehicle trajectory. It can be replaced by the coordinates of the center of mass of the vehicle. The calculation method is shown as follows [26–31]:(10)a=ap+0.5×breadth,b=bp+0.5×height.In the formula,a, b represent the coordinate value of the target centroid point of the current frame. ap,bp represent the coordinate value of the upper left vertex of the smallest enclosing rectangular box of the moving vehicle target. Breadth and height, respectively, represent the width and height of the smallest rectangular box surrounding the target. The moving speed of the vehicle target at time i is expressed by the following equation:(11)vi=ai−ai−12+bi−bi−12ti−ti−1.In the equation,ai,bi and ai−1,bi−1 represent the coordinate values of the centroid of the moving target at the current time and the previous time, respectively, ti,ti−1 represent the time interval between the current moment and the previous moment.The moving direction of the vehicle target at timei can be expressed by ∂i=arctan×da/db, where da represents the movement distance of the vehicle target on the x-axis at time i, and db represents the movement distance of the vehicle target on the y-axis at time i.


### 2.11. Detection of Abnormal Vehicle Trajectory

Taking the turning of vehicles on the highway as an example, after extracting the characteristics of turning trajectory, we apply the trained algorithm rules to classify the candidate vehicle steering trajectory events, confirm the candidate event as the final steering event, and complete the steering trajectory detection. Figure10 is a schematic diagram of the formation of the movement trajectory of the vehicle steering movement. It can be seen that the vehicle trajectory is formed by connecting multiple points through line segments, not a whole curve. The relationship between vehicle trajectory and lane line is used to realize semantic analysis of vehicle driving trajectory. When the system detects abnormal vehicle trajectory such as illegal parking, speeding, and illegal turning around, it will promptly report to the police.Figure 10
A schematic diagram of the formation of the trajectory of the vehicle steering movement.


#### 2.11.1. Vehicle Trajectory Detection and Tracking

In this research, the vehicle detection and tracking under the semantics of vehicle trajectory are mainly to separate the moving target from the background in the surveillance video obtained under 5G technology. Then, the moving vehicle target in the foreground is obtained, and the improvedKalman predictor is used to track the moving vehicle target. Figure 11 shows the original image (a) and the foreground image (b) of the multivehicle detection and tracking results on the highway. The box in the figure represents the moving vehicle detected and tracked. As can be seen from Figure 11, the system detects two cars in the 60th frame; the 75th frame continues to track the two cars previously detected. The brown car in the 81st frame of video is gradually approaching the silver car. The system has been tracking the brown car and silver car in the video from frame 85 to frame 96. Starting from frame 101, when brown car approaches silver car and then gradually leaves silver car, the system can detect the trajectory of brown car. In summary, the system can accurately detect multiple cars appearing on the highway, and the detection and tracking results are very good.Figure 11
The sum of the original image (a) and the foreground image (b) of the multivehicle detection and tracking results on the highway.
(a)(b)


#### 2.11.2. Analysis of Vehicle Trajectory and Abnormal Trajectory

In the previous section, this article tested the effect of the vehicle trajectory detection system on the detection and tracking of vehicle targets. Next, it is necessary to extract the center of mass of the vehicle target in consecutive multiple frames of images and synthesize the vehicle trajectory. Then, use the relationship between the vehicle trajectory and the lane line to analyze the characteristics of the matching vehicle trajectory type and realize the semantic analysis of the vehicle trajectory. Denoise processing for the centroid data of brown car and silver car is recorded in Figure11 to remove the abrupt trajectory points and draw the driving trajectory of the vehicle. Draw a track point every 10 frames, as shown in Figure 12. The solid line in the figure is the lane separation line, the dashed line in the figure represents the trajectory of the vehicle, and the small box on the dashed line represents the center of mass of the vehicle. Through analysis, the driving trajectory of silver car is almost parallel to the lane line, and the driving direction of the vehicle is consistent with the normal driving direction of the lane, so it can be concluded that the semantics of vehicle trajectory is normal driving (Figure 12).Figure 12
The brown car driving trajectory diagram and the silver car driving trajectory diagram in the video.In Figure12, the value of the center of mass of car changed rapidly along the Y-axis coordinate, indicating that the car speed is faster. The numerical change of the X-axis coordinate is also more obvious, indicating that the vehicle may have driving trajectories such as changing lanes. According to the recorded vehicle centroid data, the driving trajectory of the brown car is drawn (Figure 12). It can be seen that the driving direction of the car changes a lot, and there is a risk of contact with the silver car, so the driving trajectory of the brown car is abnormal, alarmed and the abnormal behaviors and violations of the silver car are snapshotted; this will help promote the management and control of intelligent transportation and reduce the occurrence of traffic accidents.In summary, this research has constructed a highway vehicle trajectory detection system under the Internet of Things and 5G technology, which realized the detection and tracking of vehicle targets in the video. The relationship between the vehicle trajectory and the lane line was used to realize the recognition of the vehicle’s driving trajectory. Detect and track moving vehicle targets through video, obtain the center of mass of the vehicle in each frame of image, synthesize the driving trajectory of the vehicle, and extract the characteristics of the vehicle in each frame of image (target position, vehicle speed, and vehicle driving direction). Then combine with the lane line equation to analyze and understand the semantics of vehicle Trajectory, and finally get the semantic representation of vehicle driving Trajectory. Finally, the results of the experimental data are summarized and analyzed. The algorithm extracted in this research can accurately perform semantic analysis on vehicle trajectory. When the system detects abnormal vehicle trajectory such as changing lanes, speeding, and turning around in violation of regulations, it can report to the police in time.


## 2.1. The Role of IoT and 5G Technology on Highway Traffic

In the context of the Internet of Things, the practical advantages of 5G communication technology are mainly reflected in the fact that this technology can increase data traffic, expand equipment, and make communication more reliable. 5G technology allows users to directly handle some network services on the network platform, saving users time and ensuring the reliability of communication services. Therefore, in the context of the Internet of Things, 5G communication technology can realize “things and things” communication services. And according to the scenarios required by the Internet of Things, the communication protocol is customized to enable the Internet of Things to have the function of the “Internet of Things.” The integration of the Internet of Things and 5G communication technology realizes the communication interconnection between vehicles and vehicles, vehicles and people, vehicles and road infrastructure, and vehicles and network service platforms (as shown in Figure1) [6]. “Vehicle to vehicle” can monitor the information between vehicles in the system in real time and realize the functions of vehicle-assisted driving, emergency collision warning, vehicle lane change and steering assist. “Vehicle and person” establishes the interconnection of information between vehicles and pedestrians, which can avoid collisions between vehicles and pedestrians and ensure the safety of pedestrians. “Vehicle and network” realizes the network interconnection between the vehicle and the network platform in the vehicle network system, and the network platform can provide navigation information and road traffic information for the vehicle. “Vehicles and road” provide basic road information for vehicles, such as road traffic conditions and speed limit information, so that violations caused by speeding vehicles can be effectively avoided.Figure 1
The Internet of Things converged 5G communication technology to get off the communication interconnection between “vehicles and vehicles,” “vehicles and person,” “vehicles and road,” and “vehicles and network service platforms.”


## 2.2. Research Status of IoT Fusion 5G in Vehicle Trajectory Detection

The highway vehicle trajectory detection system based on the Internet of Things and 5G communication technology can effectively divide the management between systems, accurately locate the physical location, and form an effective management plan so as to improve the safety of highway vehicles. Video surveillance system is one of the key elements to ensure the safety of highway vehicles. Using 5G’s ultra-high bandwidth characteristics to transmit video can realize remote monitoring of the vehicle and upload the video to the cloud for backup and storage. More and more researchers pay attention to the research of vehicle driving state detection. TheDouble Exponential Smoothing algorithm was mentioned in the literature [7], which used the actual and predicted values of traffic parameters to design and track vehicle driving signals. The Bayesian algorithm developed by Karagiannis et al. used the difference in occupancy between two adjacent detectors to determine whether traffic congestion occurs [8]. The dynamic model algorithm developed by Li et al. used the relationship between “flow-density” and “speed-density” of traffic flow to determine whether a traffic incident occurs [9]. Misbahuddin developed a low-pass filtering algorithm using a moving average method, which filters out the high-frequency and noise components in the measured traffic parameters and only retains low-frequency data to judge traffic congestion [10]. Literature [11] proposed an automatic detection method for traffic incidents based on a video-based vehicle trajectory model and established a mathematical model to determine the type of traffic incidents that occurred.


## 2.3. Internet of Things Fusion 5G Technology Used in Highway Vehicle Trajectory Detection

The most widely used sensor devices for the perception layer of the Internet of Things are barcodes and cameras. Its key technologies include radio frequency identification, barcodes, sensors, and smart embedded devices. The network layer of the Internet of Things generally uses wireless networks, the Internet, and wired networks for data transmission. The network layer mainly uses the integration of various telecommunication networks (such as 4G/5G) and the Internet to transmit data safely, completely, and quickly. The 5G technology applied to the network layer; the main technologies include ZigBee, RFID, NFC, Wi-Fi, Bluetooth, Global Positioning System (GPS) [12]. For highway vehicle monitoring, the application layer of the Internet of Things can collect vehicle information on the road through sensors such as coils and cameras. The data is transmitted to the management platform through the network, and statistics and management of information such as traffic flow, vehicle speed, and vehicle occupation ratio on the road are performed. In this research, the GPS technology and RFID technology are used. In highway traffic, GPS technology can not only provide the driver with vehicle positioning information, but also automatically select the best driving route for the driver as well as provide road traffic information and other query information. The GPS-based vehicle highway monitoring system consists of three parts: a vehicle-mounted terminal, a monitoring center, and a wireless communication network, as shown in Figure 2 [13].Figure 2
Vehicle highway monitoring system based on GPS technology.The RFID technology can automatically identify vehicle information, detect the driving route and driving status of the vehicle, collect road traffic information, provide road traffic flow data, and realize automatic charging. When the vehicle-mounted electronic tag passes the RFID reader set up on the side of the road, the RFID system will automatically record the elapsed time of the vehicle and read the basic information of the vehicle and pass it to the monitoring center. Once the monitoring center finds abnormal trajectory, it can automatically call the police and remind the staff to pay attention to the whereabouts of the vehicle and check it.


## 2.4. Research on the Highway Vehicle Driving State Detection System Model under the Framework of Internet of Things and 5G

The Internet of Things integrates 5G technology, and their interactive nodes are on-board computers. The in-vehicle system is connected through the 5G network to process data. The integration of 5G technology with the Internet of Things is the most important part of the field of intelligent transportation. Like the Internet of Things, the structure of the Internet of Things converged with 5G technology also has three levels: front-end perception layer, network transmission layer, and application layer (Figure3) [14]. The perception layer is responsible for the collection of vehicle information, and all data in the Internet of Things originates from this. With the help of 5G communication technology, it is possible to obtain the ultra-high bandwidth characteristics of the transmission video during the driving of highway vehicles. The second is the network transmission layer, which is responsible for the transmission of information. The data in the integration of the Internet of Things 5G technology can be obtained anytime and anywhere. The third is the application layer, which realizes the recognition and perception between objects and people [15].Figure 3
Architecture diagram of IoT fusion 5G technology.This research was based on the integration of 5G technology and architecture diagram of the Internet of Things, combining vehicle speed detection system and vehicle abnormal trajectory detection to form a highway trajectory driving state detection system, proposing the highway trajectory driving state detection system in this paper. The video information was transmitted through 5G ultra-high bandwidth characteristics; the video data and image data obtained by the Internet of Things were processed to realize the function of vehicle speed detection and vehicle trajectory analysis. This system was conducive to promoting the management and control of intelligent traffic on expressways and reducing the occurrence of traffic accidents. This part first introduced the semantic representation and feature analysis of vehicle traffic trajectory and then introduced the moving target detection and moving target tracking algorithms needed to extract the vehicle trajectory. Finally, the vehicle trajectory semantic analysis algorithm designed and its implementation steps were introduced, and the results of the experimental data were summarized and analyzed.


## 2.5. The Framework and Process of the Internet of Things and 5G-Integrated Highway Vehicle Driving State Detection System

If we want to realize the detection of abnormal driving vehicles on the highway, you need a complete highway trajectory driving state detection system. The system studies the perception layer, network layer, and application layer of the Internet of Things under 5G technology to build an integrated highway vehicle abnormal driving detection scheme. Based on the architecture diagram of the Internet of Things fusion 5G technology in Figure3, this study constructed the framework of the highway vehicle driving state detection system shown in Figure 4.Figure 4
The framework of the highway vehicle driving state detection system based on the integration of the Internet of Things and 5G technology.The basic goal of the framework is to decouple the real-time driving state of the vehicle from the transmission and storage of large-scale data so as to achieve the dual goals of real-time event detection and large-scale data transmission [16]. According to the dynamic trajectory of the vehicle and the available 5G communication base stations and WiFi hotspots provided by the fixed infrastructure along the road [17], we optimized and dynamically adjusted the time and place of unloading the driving data of different vehicles so as to supplement the limited transmission bandwidth provided by traditional mobile network base stations. After the vehicles are networked on a large scale, it is necessary to store the detected data according to the vehicle trajectory and provide fast retrieval services of multiple attributes. Based on this, the software module flowchart of the highway vehicle driving state detection system designed in this research is shown in Figure 5.Figure 5
The software module flowchart of the highway vehicle driving state detection system integrated with the Internet of Things 5G.


## 2.6. Research on Algorithms of Vehicle Motion Track Detection on Expressway

This research is based on the integration of the Internet of Things and 5G technology and used the video sequences collected during the driving of highway vehicles as the research object. Vehicle abnormal trajectory detection based on highway vehicle driving video generally includes three parts: moving vehicle detection, moving vehicle tracking, and abnormal trajectory recognition. Among them, the detection of moving vehicles is the basis and primary condition for the subsequent tracking of moving vehicles and the recognition of abnormal trajectories. Moving vehicle detection is the process of extracting moving vehicles by analyzing continuous images. It belongs to a kind of moving target detection, so before moving vehicle detection, this research first analyzes and researches moving target detection. At present, the detection algorithms for moving targets mainly include optical flow method, frame difference method, and background difference method [18].


### 2.6.1. Frame Difference Method

The frame difference method is an algorithm that subtracts two or more adjacent frames in a video frame image sequence to obtain the foreground area of a moving target. When there is a moving target in the video frame image, there will be obvious pixel changes in this area, but there is no pixel change phenomenon in stationary objects. Therefore, the pixels between frames are subtracted, and the pixel difference is compared with a preset threshold to determine whether there are moving objects in the image sequence, as shown in the following equation:(1)Da,b=1fia,b−fi−1a,b≥tm,0fia,b−fi−1a,b≥tm.Among them,fia,b is the image pixel of the current frame, fi−1a,b is the image pixel of the previous frame, tm is the threshold, and Da,b is the binary image pixel after the difference. Making difference between two frames. If the difference is greater than the threshold, it is the foreground; otherwise, it is the background.The method is simple and easy to implement, has low complexity, and is not sensitive to changes in the external environment such as light and weather. However, there are requirements for the speed of the vehicle. Too fast or too slow will affect the detection results of the vehicle, and it is prone to “holes” and “ghosts.”


### 2.6.2. Optical Flow Method

Optical flow field is a kind of performance movement of image gray mode. It can be calculated mainly by four methods based on gradient, matching, capability, and phase. In a frame of video image, suppose the gray value of pixela,b at time t is Fa,b,t. At time t+Δt, the pixel moves to the next new position a+Δa,b+Δb, and the corresponding gray value is Fa+Δa,b+Δb,t+Δt. Assuming dFa,b,t/dt=0 according to the consistency of the image, the brightness of the image at time t and t+Δt remains unchanged; namely, (2)Fa,b,t=Fa+Δa,b+Δb,t+Δt.Assuming that the two components of the optical flow vector of the pixel along thea, b directions are represented by u, v, respectively, and there is u=da/dt,v=db/dt, the Taylor series expansion of equation (2) is obtained:(3)Fa,b,t=Fa+Δa,b+Δb,t+Δt≈Fa,b,t+∂F∂ada+∂F∂bdb+∂F∂tdt.WhenΔt⟶0, the basic equation of optical flow can be obtained:(4)∂F∂adadt+∂F∂bdbdt+∂F∂t=Fau+Fbv+Ft=0.Among them,Fa,Fb,Ft represent the change of the pixel gray value with a, b, t, respectively.This method can accurately detect and identify the location of the moving target without knowing the scene information. At the same time, the camera is still applicable when in motion. However, this method requires a lot of calculation and time. Since changing light will be mistakenly recognized as optical flow, this method is sensitive to light and will affect the recognition effect.


### 2.6.3. Background Difference Method

The background difference method uses the difference between the current frame image and the background image and performs filtering and morphological processing on the result to obtain the foreground of the moving target, as shown in the following equations:(5)Da,b=fia,b−Bi−1a,b,(6)Fa,b=1Da,b≥tm,0Da,b≤tm.Among them,fia,b is the pixel value of the current image, Bi−1a,b is the pixel value of the background image, and Fa,b is the binarized foreground pixel value of the moving target. Through comparison, it can be seen that the background difference method is simple, and the calculation speed is fast, and a complete and accurate description of the moving target area can be obtained. It can detect both slow-moving vehicles and temporarily parked vehicles, and it meets the detection requirements of abnormal vehicle driving conditions in this paper. This method is more dependent on the background. The better the purity of the extracted background, the better the foreground effect of the detected moving target. Based on the calculation principle of the Internet of Things and 5G communication technology, this research proposes an adaptive threshold background difference method.The background difference method with adaptive threshold proposed in this study also meets the principle of the background difference method, except that the adaptive threshold algorithm is used to determine the thresholdH. This threshold will change with changes in the foreground and background, so as to optimize the extraction of moving objects. To determine the adaptive threshold H, it is necessary to calculate the average value Ddiff and standard deviation Δdiff of the interframe difference of each pixel, as follows [19]:LetIna,b represent the gray value of the nth frame image at pixel point a,b and inter represent the interval between two frames, which is set to 5 in this study. Let Fna,b be as follows:(7)Fna,b=Ina,b−Iintera,ba,b,Ddiffa,b=1M∑interMFa,b,Δdiffa,b=1M∑interM∑Fna,b−Ddiffa,b2.In order to ensure the accuracy ofDdiff and Δdiff, M is usually large enough. In this study, M is set to 35+inter. After obtaining Ddiff and Δdiff, the threshold H can be determined as follows: (8)H=Ddiff+βΔdiff.After the detection of each frame of image, the background model must be updated. For all pixelsa,b, let Da,b,Ddiffa,b,Δdiffa,b be, respectively, D′a,b,Ddiff′a,b,Δdiff′a,b after update, which are shown in the following equations:(9)D′a,b=I−αDa,b+αIa,b,Ddiff′a,b=I−αDdiffa,b+αFa,b,Δdiff′a,b=I−αDdiffa,b+αFa,b−D′a,b.


## 2.6.1. Frame Difference Method

The frame difference method is an algorithm that subtracts two or more adjacent frames in a video frame image sequence to obtain the foreground area of a moving target. When there is a moving target in the video frame image, there will be obvious pixel changes in this area, but there is no pixel change phenomenon in stationary objects. Therefore, the pixels between frames are subtracted, and the pixel difference is compared with a preset threshold to determine whether there are moving objects in the image sequence, as shown in the following equation:(1)Da,b=1fia,b−fi−1a,b≥tm,0fia,b−fi−1a,b≥tm.Among them,fia,b is the image pixel of the current frame, fi−1a,b is the image pixel of the previous frame, tm is the threshold, and Da,b is the binary image pixel after the difference. Making difference between two frames. If the difference is greater than the threshold, it is the foreground; otherwise, it is the background.The method is simple and easy to implement, has low complexity, and is not sensitive to changes in the external environment such as light and weather. However, there are requirements for the speed of the vehicle. Too fast or too slow will affect the detection results of the vehicle, and it is prone to “holes” and “ghosts.”


## 2.6.2. Optical Flow Method

Optical flow field is a kind of performance movement of image gray mode. It can be calculated mainly by four methods based on gradient, matching, capability, and phase. In a frame of video image, suppose the gray value of pixela,b at time t is Fa,b,t. At time t+Δt, the pixel moves to the next new position a+Δa,b+Δb, and the corresponding gray value is Fa+Δa,b+Δb,t+Δt. Assuming dFa,b,t/dt=0 according to the consistency of the image, the brightness of the image at time t and t+Δt remains unchanged; namely, (2)Fa,b,t=Fa+Δa,b+Δb,t+Δt.Assuming that the two components of the optical flow vector of the pixel along thea, b directions are represented by u, v, respectively, and there is u=da/dt,v=db/dt, the Taylor series expansion of equation (2) is obtained:(3)Fa,b,t=Fa+Δa,b+Δb,t+Δt≈Fa,b,t+∂F∂ada+∂F∂bdb+∂F∂tdt.WhenΔt⟶0, the basic equation of optical flow can be obtained:(4)∂F∂adadt+∂F∂bdbdt+∂F∂t=Fau+Fbv+Ft=0.Among them,Fa,Fb,Ft represent the change of the pixel gray value with a, b, t, respectively.This method can accurately detect and identify the location of the moving target without knowing the scene information. At the same time, the camera is still applicable when in motion. However, this method requires a lot of calculation and time. Since changing light will be mistakenly recognized as optical flow, this method is sensitive to light and will affect the recognition effect.


## 2.6.3. Background Difference Method

The background difference method uses the difference between the current frame image and the background image and performs filtering and morphological processing on the result to obtain the foreground of the moving target, as shown in the following equations:(5)Da,b=fia,b−Bi−1a,b,(6)Fa,b=1Da,b≥tm,0Da,b≤tm.Among them,fia,b is the pixel value of the current image, Bi−1a,b is the pixel value of the background image, and Fa,b is the binarized foreground pixel value of the moving target. Through comparison, it can be seen that the background difference method is simple, and the calculation speed is fast, and a complete and accurate description of the moving target area can be obtained. It can detect both slow-moving vehicles and temporarily parked vehicles, and it meets the detection requirements of abnormal vehicle driving conditions in this paper. This method is more dependent on the background. The better the purity of the extracted background, the better the foreground effect of the detected moving target. Based on the calculation principle of the Internet of Things and 5G communication technology, this research proposes an adaptive threshold background difference method.The background difference method with adaptive threshold proposed in this study also meets the principle of the background difference method, except that the adaptive threshold algorithm is used to determine the thresholdH. This threshold will change with changes in the foreground and background, so as to optimize the extraction of moving objects. To determine the adaptive threshold H, it is necessary to calculate the average value Ddiff and standard deviation Δdiff of the interframe difference of each pixel, as follows [19]:LetIna,b represent the gray value of the nth frame image at pixel point a,b and inter represent the interval between two frames, which is set to 5 in this study. Let Fna,b be as follows:(7)Fna,b=Ina,b−Iintera,ba,b,Ddiffa,b=1M∑interMFa,b,Δdiffa,b=1M∑interM∑Fna,b−Ddiffa,b2.In order to ensure the accuracy ofDdiff and Δdiff, M is usually large enough. In this study, M is set to 35+inter. After obtaining Ddiff and Δdiff, the threshold H can be determined as follows: (8)H=Ddiff+βΔdiff.After the detection of each frame of image, the background model must be updated. For all pixelsa,b, let Da,b,Ddiffa,b,Δdiffa,b be, respectively, D′a,b,Ddiff′a,b,Δdiff′a,b after update, which are shown in the following equations:(9)D′a,b=I−αDa,b+αIa,b,Ddiff′a,b=I−αDdiffa,b+αFa,b,Δdiff′a,b=I−αDdiffa,b+αFa,b−D′a,b.


## 2.7. Stable Tracking of the Driving State of Highway Vehicles with the Integration of the Internet of Things and 5G

The process of vehicle tracking is actually the process of feature matching, so it is necessary to select the appropriate feature and the optimal matching strategy. There are many characteristics of vehicles, including regional blocks, gradients, textures, depth, and optical flow [20]. The matching methods for selecting different features are also different, and their time and space complexity are also different. Sometimes, in order to increase the accuracy of matching, it is difficult to realize it, and it is necessary to continuously track the vehicle. The perception layer under the integration of the Internet of Things 5G technology only needs to use a high-definition camera to collect the data required by the system. Therefore, the highway vehicle abnormal trajectory detection method based on high-quality video can not only obtain real-time traffic status information more intuitively and comprehensively, but also has higher reliability [21]. This article started from the perspective of video, through real-time analysis of vehicle video, detected and tracked related vehicles, and detected vehicles with abnormal trajectories in time. As shown in Figure 6, a video-based vehicle abnormal trajectory detection system usually includes a camera, an optical transceiver, a communication device, a video detection processing unit, and a display unit. The camera is responsible for collecting video in real time; the video detection processing unit is responsible for the processing of the vehicle video, extracting useful information of the vehicle from it, and identifying the vehicle with abnormal trajectory. The data computer is used to control the system and store information about vehicles that have abnormal trajectories.Figure 6
High-speed vehicle abnormal driving detection system structure based on high-definition video under the background of the integration of the Internet of Things and 5G.The camera is generally set directly above or on the side of a road in a city, and the height is generally between 5 and 10 meters. The schematic diagram and example of the spatial position of the camera and the road vehicle are shown in Figure7 [22].Figure 7
Schematic diagram of the spatial position of the camera and road vehicles (a) and example (b).
(a)(b)At the same time, in order to easily obtain the actual displacement of the vehicle movement, it is necessary to understand the unified standards of the lane lines on the roads in China. The highway is divided into multiple lanes by dashed white lines. Among them, the length and width of the white solid line in the dashed white line and the longitudinal interval between the two white performances are fixed, and the width of the lane divided by the lane line is also fixed, with a value of 375 cm [23]. The standard maps of the Chinese highway lane lines are shown in Figure 8(a). Because of the vehicle speed detection in this study, all the data obtained comes from the camera. Therefore, the standard map of the Chinese highway lane lines will be seen through and become a trapezoid-like shape (Figure 8(b)). The vehicle coordinates obtained by processing the video data are pixel coordinates and need to be converted into actual physical coordinates to obtain the actual displacement of the vehicle movement. The semantic analysis of vehicle trajectory in this article combines the relationship between vehicle trajectory and lane lines to analyze and derive the semantics of vehicle trajectory. Therefore, firstly, it is necessary to label the lane segments separately for the background images in the experimental data single-lane traffic video and multilane traffic video. The labeling result is shown in Figure 8(d), where the red line segment is the lane line labeling.Figure 8
The standard map example (a) and schematic diagram (b) of the Chinese expressway lane lines, the real picture (c), and the labeling result map of the lane lines (d).
(a)(b)(c)(d)


## 2.8. The Abnormal Recognition of the Driving State of Highway Vehicles Based on the Integration of the Internet of Things and 5G

Recognition of abnormal trajectory of vehicles on highways is a complex process, involving trajectory learning and trajectory discrimination, which requires high recognition stability and robustness [24]. Therefore, this article mainly focuses on abnormal driving trajectories such as U-turn, continuous lane change, obstacle avoidance, and emergency braking in the highway environment. These trajectories often indicate that there may be abnormal traffic conditions, or they may cause accidents themselves.


## 2.9. Semantic Representation of Abnormal Vehicle Trajectory

The vehicle trajectory obtained from the highway vehicle abnormality detection system can be used to analyze and identify vehicle driving trajectory [25]. Through a lot of observation and analysis, this article divides the vehicle driving trajectory into several simple semantic representations, such as normal driving, retrograde, stop driving, U-turn and retrograde, lane change, overtaking, etc. The semantic representation of vehicle trajectory is shown in Figure 9, among them, (a) is the trajectory of normal driving, (b) is the trajectory of reverse, (c) is the trajectory of parking, (d) is the trajectory of changing lanes, and (e) is the trajectory of overtaking. After the vehicle’s trajectory is obtained through the moving object detection and moving object tracking algorithm, in order to realize the semantic analysis of the vehicle driving, it is first necessary to extract the movement characteristics of the vehicle trajectory. Only by selecting appropriate motion characteristics can the vehicle trajectory be accurately described. Commonly used movement characteristics are target position, movement speed, and movement direction.Figure 9
Schematic diagram of the semantic expression of the vehicle trajectory used in this study. (a) Trajectory of normal driving, (b) trajectory of reverse, (c) trajectory of parking, (d) trajectory of changing lanes, and (e) trajectory of overtaking.
(a)(b)(c)(d)(e)


## 2.10. Motion Feature Extraction of Vehicle Trajectory

To realize the semantic analysis of the vehicle trajectory, it is necessary to extract the motion characteristics of the vehicle trajectory. Commonly used motion characteristics are target position, motion speed, and motion direction. The information of the target position of the vehicle is an important indicator for semantic analysis and understanding of vehicle trajectory. It can be replaced by the coordinates of the center of mass of the vehicle. The calculation method is shown as follows [26–31]:(10)a=ap+0.5×breadth,b=bp+0.5×height.In the formula,a, b represent the coordinate value of the target centroid point of the current frame. ap,bp represent the coordinate value of the upper left vertex of the smallest enclosing rectangular box of the moving vehicle target. Breadth and height, respectively, represent the width and height of the smallest rectangular box surrounding the target. The moving speed of the vehicle target at time i is expressed by the following equation:(11)vi=ai−ai−12+bi−bi−12ti−ti−1.In the equation,ai,bi and ai−1,bi−1 represent the coordinate values of the centroid of the moving target at the current time and the previous time, respectively, ti,ti−1 represent the time interval between the current moment and the previous moment.The moving direction of the vehicle target at timei can be expressed by ∂i=arctan×da/db, where da represents the movement distance of the vehicle target on the x-axis at time i, and db represents the movement distance of the vehicle target on the y-axis at time i.


## 2.11. Detection of Abnormal Vehicle Trajectory

Taking the turning of vehicles on the highway as an example, after extracting the characteristics of turning trajectory, we apply the trained algorithm rules to classify the candidate vehicle steering trajectory events, confirm the candidate event as the final steering event, and complete the steering trajectory detection. Figure10 is a schematic diagram of the formation of the movement trajectory of the vehicle steering movement. It can be seen that the vehicle trajectory is formed by connecting multiple points through line segments, not a whole curve. The relationship between vehicle trajectory and lane line is used to realize semantic analysis of vehicle driving trajectory. When the system detects abnormal vehicle trajectory such as illegal parking, speeding, and illegal turning around, it will promptly report to the police.Figure 10
A schematic diagram of the formation of the trajectory of the vehicle steering movement.


### 2.11.1. Vehicle Trajectory Detection and Tracking

In this research, the vehicle detection and tracking under the semantics of vehicle trajectory are mainly to separate the moving target from the background in the surveillance video obtained under 5G technology. Then, the moving vehicle target in the foreground is obtained, and the improvedKalman predictor is used to track the moving vehicle target. Figure 11 shows the original image (a) and the foreground image (b) of the multivehicle detection and tracking results on the highway. The box in the figure represents the moving vehicle detected and tracked. As can be seen from Figure 11, the system detects two cars in the 60th frame; the 75th frame continues to track the two cars previously detected. The brown car in the 81st frame of video is gradually approaching the silver car. The system has been tracking the brown car and silver car in the video from frame 85 to frame 96. Starting from frame 101, when brown car approaches silver car and then gradually leaves silver car, the system can detect the trajectory of brown car. In summary, the system can accurately detect multiple cars appearing on the highway, and the detection and tracking results are very good.Figure 11
The sum of the original image (a) and the foreground image (b) of the multivehicle detection and tracking results on the highway.
(a)(b)


### 2.11.2. Analysis of Vehicle Trajectory and Abnormal Trajectory

In the previous section, this article tested the effect of the vehicle trajectory detection system on the detection and tracking of vehicle targets. Next, it is necessary to extract the center of mass of the vehicle target in consecutive multiple frames of images and synthesize the vehicle trajectory. Then, use the relationship between the vehicle trajectory and the lane line to analyze the characteristics of the matching vehicle trajectory type and realize the semantic analysis of the vehicle trajectory. Denoise processing for the centroid data of brown car and silver car is recorded in Figure11 to remove the abrupt trajectory points and draw the driving trajectory of the vehicle. Draw a track point every 10 frames, as shown in Figure 12. The solid line in the figure is the lane separation line, the dashed line in the figure represents the trajectory of the vehicle, and the small box on the dashed line represents the center of mass of the vehicle. Through analysis, the driving trajectory of silver car is almost parallel to the lane line, and the driving direction of the vehicle is consistent with the normal driving direction of the lane, so it can be concluded that the semantics of vehicle trajectory is normal driving (Figure 12).Figure 12
The brown car driving trajectory diagram and the silver car driving trajectory diagram in the video.In Figure12, the value of the center of mass of car changed rapidly along the Y-axis coordinate, indicating that the car speed is faster. The numerical change of the X-axis coordinate is also more obvious, indicating that the vehicle may have driving trajectories such as changing lanes. According to the recorded vehicle centroid data, the driving trajectory of the brown car is drawn (Figure 12). It can be seen that the driving direction of the car changes a lot, and there is a risk of contact with the silver car, so the driving trajectory of the brown car is abnormal, alarmed and the abnormal behaviors and violations of the silver car are snapshotted; this will help promote the management and control of intelligent transportation and reduce the occurrence of traffic accidents.In summary, this research has constructed a highway vehicle trajectory detection system under the Internet of Things and 5G technology, which realized the detection and tracking of vehicle targets in the video. The relationship between the vehicle trajectory and the lane line was used to realize the recognition of the vehicle’s driving trajectory. Detect and track moving vehicle targets through video, obtain the center of mass of the vehicle in each frame of image, synthesize the driving trajectory of the vehicle, and extract the characteristics of the vehicle in each frame of image (target position, vehicle speed, and vehicle driving direction). Then combine with the lane line equation to analyze and understand the semantics of vehicle Trajectory, and finally get the semantic representation of vehicle driving Trajectory. Finally, the results of the experimental data are summarized and analyzed. The algorithm extracted in this research can accurately perform semantic analysis on vehicle trajectory. When the system detects abnormal vehicle trajectory such as changing lanes, speeding, and turning around in violation of regulations, it can report to the police in time.


## 2.11.1. Vehicle Trajectory Detection and Tracking

In this research, the vehicle detection and tracking under the semantics of vehicle trajectory are mainly to separate the moving target from the background in the surveillance video obtained under 5G technology. Then, the moving vehicle target in the foreground is obtained, and the improvedKalman predictor is used to track the moving vehicle target. Figure 11 shows the original image (a) and the foreground image (b) of the multivehicle detection and tracking results on the highway. The box in the figure represents the moving vehicle detected and tracked. As can be seen from Figure 11, the system detects two cars in the 60th frame; the 75th frame continues to track the two cars previously detected. The brown car in the 81st frame of video is gradually approaching the silver car. The system has been tracking the brown car and silver car in the video from frame 85 to frame 96. Starting from frame 101, when brown car approaches silver car and then gradually leaves silver car, the system can detect the trajectory of brown car. In summary, the system can accurately detect multiple cars appearing on the highway, and the detection and tracking results are very good.Figure 11
The sum of the original image (a) and the foreground image (b) of the multivehicle detection and tracking results on the highway.
(a)(b)


## 2.11.2. Analysis of Vehicle Trajectory and Abnormal Trajectory

In the previous section, this article tested the effect of the vehicle trajectory detection system on the detection and tracking of vehicle targets. Next, it is necessary to extract the center of mass of the vehicle target in consecutive multiple frames of images and synthesize the vehicle trajectory. Then, use the relationship between the vehicle trajectory and the lane line to analyze the characteristics of the matching vehicle trajectory type and realize the semantic analysis of the vehicle trajectory. Denoise processing for the centroid data of brown car and silver car is recorded in Figure11 to remove the abrupt trajectory points and draw the driving trajectory of the vehicle. Draw a track point every 10 frames, as shown in Figure 12. The solid line in the figure is the lane separation line, the dashed line in the figure represents the trajectory of the vehicle, and the small box on the dashed line represents the center of mass of the vehicle. Through analysis, the driving trajectory of silver car is almost parallel to the lane line, and the driving direction of the vehicle is consistent with the normal driving direction of the lane, so it can be concluded that the semantics of vehicle trajectory is normal driving (Figure 12).Figure 12
The brown car driving trajectory diagram and the silver car driving trajectory diagram in the video.In Figure12, the value of the center of mass of car changed rapidly along the Y-axis coordinate, indicating that the car speed is faster. The numerical change of the X-axis coordinate is also more obvious, indicating that the vehicle may have driving trajectories such as changing lanes. According to the recorded vehicle centroid data, the driving trajectory of the brown car is drawn (Figure 12). It can be seen that the driving direction of the car changes a lot, and there is a risk of contact with the silver car, so the driving trajectory of the brown car is abnormal, alarmed and the abnormal behaviors and violations of the silver car are snapshotted; this will help promote the management and control of intelligent transportation and reduce the occurrence of traffic accidents.In summary, this research has constructed a highway vehicle trajectory detection system under the Internet of Things and 5G technology, which realized the detection and tracking of vehicle targets in the video. The relationship between the vehicle trajectory and the lane line was used to realize the recognition of the vehicle’s driving trajectory. Detect and track moving vehicle targets through video, obtain the center of mass of the vehicle in each frame of image, synthesize the driving trajectory of the vehicle, and extract the characteristics of the vehicle in each frame of image (target position, vehicle speed, and vehicle driving direction). Then combine with the lane line equation to analyze and understand the semantics of vehicle Trajectory, and finally get the semantic representation of vehicle driving Trajectory. Finally, the results of the experimental data are summarized and analyzed. The algorithm extracted in this research can accurately perform semantic analysis on vehicle trajectory. When the system detects abnormal vehicle trajectory such as changing lanes, speeding, and turning around in violation of regulations, it can report to the police in time.


## 3. Conclusion

The combination of 5G mobile network communication technology, the Internet of Things equipment, and artificial intelligence is the main direction for the development of information technology in the future. From the perspective of the wireless communication and transmission of the Internet of Things and the car, it is to open up the sharing of data and information between people, vehicles, and the external environment and build a multi-interconnected, integrated control business service network. From the perspective of application scenarios of 5G communication and Internet of Things, cars, car driving route planning, vehicle information interaction, congestion monitoring, exhaust pollution monitoring, etc., can all use 5G communication and the Internet of Things technology to target the traffic driving intention of the car driver. Dynamic traffic information of surrounding vehicles is automatically perceived. With the support of the Internet of Things and 5G technology, this research divides the driving trajectory of vehicles on the highway into several simple semantic representations through a large number of observations and analysis and conducts feature analysis on the semantic representations. Then, it introduces the moving target detection and moving target tracking algorithms needed to extract the vehicle trajectory and designs a semantic analysis algorithm of vehicle trajectory that combines the vehicle trajectory and lane lines. Through video detection and tracking of moving vehicle targets, the vehicle’s driving trajectory can be obtained, and the vehicle’s movement characteristics in each frame of image can be extracted, including vehicle position, vehicle speed, and vehicle driving direction. According to the relationship between the trajectory of the vehicle and the lane line, the semantic analysis of vehicle trajectory is realized, and then it is judged whether the vehicle has violated regulations.



---

*Source: 9961428-2021-04-28.xml*