# The Key Technologies of Marine Multiobjective Ship Monitoring and Tracking Based on Computer Vision

Authors: Qiang Li; Li Yang
Journal: Mobile Information Systems
 (2022)

---

## Abstract

In recent years, one of the key research contents of computer vision is to realize the online monitoring and information tracking of unknown targets by sea ships. For multitarget ships at sea, the most important thing is the information tracking direction. It is very appropriate to apply this computer vision technology to ship monitoring at sea. This paper uses the ship target detection algorithm (STDA) to study the tracking and detection of sea multitarget ships based on computer vision to be applied in practice and improve the protection and safety of sea navigation. Based on the background of ocean intelligent monitoring, the image processing algorithm of key target detection and tracking in the ship detection and tracking system is studied, and the ocean monitoring video image is obtained by installing the detection and tracking effect of the shipborne camera verification system. The ship object detection algorithm (STDA) of the sea antenna is compared with the traditional target detection algorithm. Experimental results show that the proposed algorithm can effectively eliminate the noise interference caused by the waves and the sky, with strong accuracy and real-time performance. The research results of this paper can effectively compensate for the shortcomings of the traditional TLD multitarget ship tracking algorithm and make its speed and real-time performance greatly meet the requirements of ship monitoring. At the same time, it improves the marine monitoring capability to provide reliable technical support for the development of intelligent ships.


---

## Body

## 1. Introduction

China is a maritime power. In order to respond to the call of building China into a maritime power, protect China’s marine field, and timely complete the tasks of maritime rights protection and maritime rescue, we need to improve the ability of marine monitoring and control and focus on the development of target monitoring equipment in the marine field. Therefore, the research on maritime long-range target detection algorithm has certain theoretical and practical significance [1, 2]. Catadioptric panoramic vision system can obtain high-resolution scene information at one time to meet the requirements of long-distance and large field of view in the field of marine monitoring. However, the panoramic sea area image collected by the system is generally visible light image. At present, the small target detection technology based on visible light image is not mature; therefore, this paper proposes a panoramic vision-based maritime vision target detection algorithm [3, 4].For the application of computer vision in tracking and recognition, many scholars have made extraordinary achievements [5, 6]. For example, a graduate student from a university in China proposed an image denoising method, which tracks the target in video images through OPENCV, uses the contour detection method for contour detection, finds the largest contour in the image, and predicts the location of moving targets combined with the interactive Kalman filter method. Moving targets within the video range were then tracked by calculating the cockpit and the heading direction of the plane [7, 8]. In addition, some scholars mainly focus on exploring an accurate, robust, and real-time marine visual tracking system for ships sailing in various key waters covered by inland river CCTV system [9, 10].This paper systematically analyzes the characteristics of target detection and tracking image processing algorithms involved in sea surface monitoring. Around the research focus of “how to effectively detect and track the abnormal behavior of suspicious ships in the complex and changeable sea environment and close range ships in real time and accurately,” some technical improvements and innovative algorithms are proposed, and these key technologies are compared with the traditional detection and monitoring technology to verify its advantages in real-time and accuracy [11, 12]. With the continuous improvement of the status of marine in national economic development and national defense security, it is increasingly important to monitor marine timely and accurate targets [13].The information perception technology of computer vision can effectively make up for the shortcomings of marine radar and automatic identification system in marine target detection and tracking and ensure the safety of ship navigation. The improved TLD multitarget ship tracking algorithm takes the least time per frame, improves the processing efficiency, and makes the speed and real-time performance greatly meet the requirements of ship monitoring. At the same time, it saves the time for the handling of emergencies and increases the technical support for the development of intelligent ships.


## 2. Computer Vision and Marine Ship Monitoring and Tracking Technology

### 2.1. Overview of Key Technologies of Ship Identification

Ship image classification algorithm can be roughly divided into two categories. One is the need for manual ship classification, the other is the automatic classification of the corresponding ship by computer. How to enhance the classification and recognition level of classifiers has become the focus of image recognition research. In this paper, we need to train the corresponding classifiers to guide their learning.


### 2.2. Target Detection and Tracking

#### 2.2.1. Interframe Difference Method

The interframe difference method can set a nonzero threshold according to the actual situation. Then, the binary image of the image is obtained by comparing with the set threshold, so as to analyze the motion characteristics of the continuous image sequence and then determine whether there is a moving target in the continuous image sequence. Interframe difference method not only has good adaptability for static targets but also is very effective for dynamic targets. The formula for calculating the difference image of adjacent frames is as follows:(1)Dkx,y=fkx,y−fk+1x,y,wherex and y are the discrete coordinates of the image; assuming that the set threshold value is t, the difference image Dkx,y is compared with the set threshold t(2)Rkx,y=1,Dkx,y>T,0,Dkx,y<T.Through the above steps, we can realize the binarization of the difference image, so as to achieve the function of moving region image.


#### 2.2.2. Optical Flow Method

When the optical flow method is used to detect the moving object, the position of the moving object will change with the time. So the optical flow changes with time. The disadvantage of optical flow computing is that it has a large amount of computation and is easy to produce delay when processing real-time images, which leads to poor accuracy and real-time performance.


#### 2.2.3. Background Difference Method

The principle of background subtraction method is to first calculate the absolute value of the difference between the background image and the currentk-th frame image. Then, the difference image is binarized and filtered by morphology. The calculation formula of background difference method is as follows:
(3)Dkx,y=fkx,y−fbkx,y.Unlike the frame difference method,fbkx,y is a background image.


#### 2.2.4. Ship Target Detection Algorithm (STDA) Based on Sea Antenna

The data processing object of the detection module in the marine ship detection and tracking system is the video stream of the marine ships. The purpose is to detect the ships in the field of vision, count the number of the corresponding ships, extract the ship target, and prepare for the following ship tracking and classification recognition.(1) Sea Antenna Extraction Based on Otsu Method and Hough Transform. In this paper, the sea antenna is extracted by maximum interclass variance and Hough transform, which is helpful to remove the noise outside the sea antenna. According to the background characteristics of offshore environment, the maximum interclass variance segmentation method and the famous Hough transform are used to detect and extract the sea antenna. Then, the image is detected by the position coordinates of the sea antenna, so as to reduce the traversal time of the image and improve the real-time performance of the detection. The formula of the maximum interclass variance method is as follows:
(4)w0=N0M×N.(2) Delimitation of Detection Area and Morphological Filtering Operation. Through Shi Tomasi corner detection, we can know the preliminary position of the ship. Then, combined with the position of the sea antenna, the ship target detection area is defined, that is, the position of the ship in the detection area. Combined with the actual situation, corner detection can be carried out near the Haitian line, and then, a straight line can be drawn according to the position of the corner at the top and 15 pixels, and a straight line can be drawn at the position of the corner at the bottom and 15 pixels. The area composed of these two lines is the ship target detection area, which is convenient for later to calculate the threshold value in the delimited area by using the maximum interclass variance; finally, the preliminary extraction of the corresponding ship targets is completed. After recalculation of the threshold value of the image in the delimited area, the new threshold value is 0.1824 calculated by using the formula of maximum interclass variance calculation. The threshold segmentation can be used to get the ship target image.


### 2.3. Multitarget Ship Tracking

The tracking of multiobjective marine ships based on video is the most important step in identifying all the key steps of marine ships. Good tracking algorithm can track the ships accurately and provide the personnel with important navigation information such as the direction of navigation and the relevant coordinate position of suspicious ships for the personnel on board, whether in the management of marine supervision or the judgment of civil ships, all of them have a very wide application prospect. In this paper, a ship tracking algorithm based on improved TLD is proposed. The real-time performance and accuracy of the three algorithms are discussed by comparing with the two popular tracking algorithms.


## 2.1. Overview of Key Technologies of Ship Identification

Ship image classification algorithm can be roughly divided into two categories. One is the need for manual ship classification, the other is the automatic classification of the corresponding ship by computer. How to enhance the classification and recognition level of classifiers has become the focus of image recognition research. In this paper, we need to train the corresponding classifiers to guide their learning.


## 2.2. Target Detection and Tracking

### 2.2.1. Interframe Difference Method

The interframe difference method can set a nonzero threshold according to the actual situation. Then, the binary image of the image is obtained by comparing with the set threshold, so as to analyze the motion characteristics of the continuous image sequence and then determine whether there is a moving target in the continuous image sequence. Interframe difference method not only has good adaptability for static targets but also is very effective for dynamic targets. The formula for calculating the difference image of adjacent frames is as follows:(1)Dkx,y=fkx,y−fk+1x,y,wherex and y are the discrete coordinates of the image; assuming that the set threshold value is t, the difference image Dkx,y is compared with the set threshold t(2)Rkx,y=1,Dkx,y>T,0,Dkx,y<T.Through the above steps, we can realize the binarization of the difference image, so as to achieve the function of moving region image.


### 2.2.2. Optical Flow Method

When the optical flow method is used to detect the moving object, the position of the moving object will change with the time. So the optical flow changes with time. The disadvantage of optical flow computing is that it has a large amount of computation and is easy to produce delay when processing real-time images, which leads to poor accuracy and real-time performance.


### 2.2.3. Background Difference Method

The principle of background subtraction method is to first calculate the absolute value of the difference between the background image and the currentk-th frame image. Then, the difference image is binarized and filtered by morphology. The calculation formula of background difference method is as follows:
(3)Dkx,y=fkx,y−fbkx,y.Unlike the frame difference method,fbkx,y is a background image.


### 2.2.4. Ship Target Detection Algorithm (STDA) Based on Sea Antenna

The data processing object of the detection module in the marine ship detection and tracking system is the video stream of the marine ships. The purpose is to detect the ships in the field of vision, count the number of the corresponding ships, extract the ship target, and prepare for the following ship tracking and classification recognition.(1) Sea Antenna Extraction Based on Otsu Method and Hough Transform. In this paper, the sea antenna is extracted by maximum interclass variance and Hough transform, which is helpful to remove the noise outside the sea antenna. According to the background characteristics of offshore environment, the maximum interclass variance segmentation method and the famous Hough transform are used to detect and extract the sea antenna. Then, the image is detected by the position coordinates of the sea antenna, so as to reduce the traversal time of the image and improve the real-time performance of the detection. The formula of the maximum interclass variance method is as follows:
(4)w0=N0M×N.(2) Delimitation of Detection Area and Morphological Filtering Operation. Through Shi Tomasi corner detection, we can know the preliminary position of the ship. Then, combined with the position of the sea antenna, the ship target detection area is defined, that is, the position of the ship in the detection area. Combined with the actual situation, corner detection can be carried out near the Haitian line, and then, a straight line can be drawn according to the position of the corner at the top and 15 pixels, and a straight line can be drawn at the position of the corner at the bottom and 15 pixels. The area composed of these two lines is the ship target detection area, which is convenient for later to calculate the threshold value in the delimited area by using the maximum interclass variance; finally, the preliminary extraction of the corresponding ship targets is completed. After recalculation of the threshold value of the image in the delimited area, the new threshold value is 0.1824 calculated by using the formula of maximum interclass variance calculation. The threshold segmentation can be used to get the ship target image.


## 2.2.1. Interframe Difference Method

The interframe difference method can set a nonzero threshold according to the actual situation. Then, the binary image of the image is obtained by comparing with the set threshold, so as to analyze the motion characteristics of the continuous image sequence and then determine whether there is a moving target in the continuous image sequence. Interframe difference method not only has good adaptability for static targets but also is very effective for dynamic targets. The formula for calculating the difference image of adjacent frames is as follows:(1)Dkx,y=fkx,y−fk+1x,y,wherex and y are the discrete coordinates of the image; assuming that the set threshold value is t, the difference image Dkx,y is compared with the set threshold t(2)Rkx,y=1,Dkx,y>T,0,Dkx,y<T.Through the above steps, we can realize the binarization of the difference image, so as to achieve the function of moving region image.


## 2.2.2. Optical Flow Method

When the optical flow method is used to detect the moving object, the position of the moving object will change with the time. So the optical flow changes with time. The disadvantage of optical flow computing is that it has a large amount of computation and is easy to produce delay when processing real-time images, which leads to poor accuracy and real-time performance.


## 2.2.3. Background Difference Method

The principle of background subtraction method is to first calculate the absolute value of the difference between the background image and the currentk-th frame image. Then, the difference image is binarized and filtered by morphology. The calculation formula of background difference method is as follows:
(3)Dkx,y=fkx,y−fbkx,y.Unlike the frame difference method,fbkx,y is a background image.


## 2.2.4. Ship Target Detection Algorithm (STDA) Based on Sea Antenna

The data processing object of the detection module in the marine ship detection and tracking system is the video stream of the marine ships. The purpose is to detect the ships in the field of vision, count the number of the corresponding ships, extract the ship target, and prepare for the following ship tracking and classification recognition.(1) Sea Antenna Extraction Based on Otsu Method and Hough Transform. In this paper, the sea antenna is extracted by maximum interclass variance and Hough transform, which is helpful to remove the noise outside the sea antenna. According to the background characteristics of offshore environment, the maximum interclass variance segmentation method and the famous Hough transform are used to detect and extract the sea antenna. Then, the image is detected by the position coordinates of the sea antenna, so as to reduce the traversal time of the image and improve the real-time performance of the detection. The formula of the maximum interclass variance method is as follows:
(4)w0=N0M×N.(2) Delimitation of Detection Area and Morphological Filtering Operation. Through Shi Tomasi corner detection, we can know the preliminary position of the ship. Then, combined with the position of the sea antenna, the ship target detection area is defined, that is, the position of the ship in the detection area. Combined with the actual situation, corner detection can be carried out near the Haitian line, and then, a straight line can be drawn according to the position of the corner at the top and 15 pixels, and a straight line can be drawn at the position of the corner at the bottom and 15 pixels. The area composed of these two lines is the ship target detection area, which is convenient for later to calculate the threshold value in the delimited area by using the maximum interclass variance; finally, the preliminary extraction of the corresponding ship targets is completed. After recalculation of the threshold value of the image in the delimited area, the new threshold value is 0.1824 calculated by using the formula of maximum interclass variance calculation. The threshold segmentation can be used to get the ship target image.


## 2.3. Multitarget Ship Tracking

The tracking of multiobjective marine ships based on video is the most important step in identifying all the key steps of marine ships. Good tracking algorithm can track the ships accurately and provide the personnel with important navigation information such as the direction of navigation and the relevant coordinate position of suspicious ships for the personnel on board, whether in the management of marine supervision or the judgment of civil ships, all of them have a very wide application prospect. In this paper, a ship tracking algorithm based on improved TLD is proposed. The real-time performance and accuracy of the three algorithms are discussed by comparing with the two popular tracking algorithms.


## 3. The Experiment of STDA Based on Sea Antenna

### 3.1. Experimental Materials and Environment

The video material of the experiment is the video image taken by the shipboard camera in the sea area of our province. The running environment of the detection algorithm is Matlab 2016rb and VS2010 software environment. The CPU of the computer is Intel Core i7, the memory size is 16g, and the operating system is Windows 10. Before target detection, the video image size is adjusted to320∗240. After image graying and filtering, the STDA based on sea antenna is tested.


### 3.2. Experimental Methods

The control experiment was used in this experiment. In order to verify the STDA based on sea antenna, this paper uses the same group of video sequences for experiments and compares the experimental results of four different algorithms of target detection and tracking.


### 3.3. Experimental Steps

First of all, we use the interframe difference method, optical flow method, background difference method, and sea antenna-based remote ship target monitoring method to process the remote ship target and record the detection time, analyze and compare the difference between them and the original image, and compare the detection time of each method. Then, the collected video is detected and verified, and a certain continuous time period of a day is selected from the collected data for verification. The period from 9:00 a.m. to 2:00 p.m. on December 12, 2020, is selected to test the statistical effect of the detection module. Because the number of ships in the open sea is relatively small, the time interval set here is 1 hour. Finally, in order to make a more accurate evaluation of the ship tracking effect, we need to calculate and analyze the ship tracking effect. At this time, we introduce two evaluation indexes, namely, the speed index and the accuracy index. Through the calculation of these two indexes, we can more comprehensively understand and analyze the performance of the tracking algorithm.


## 3.1. Experimental Materials and Environment

The video material of the experiment is the video image taken by the shipboard camera in the sea area of our province. The running environment of the detection algorithm is Matlab 2016rb and VS2010 software environment. The CPU of the computer is Intel Core i7, the memory size is 16g, and the operating system is Windows 10. Before target detection, the video image size is adjusted to320∗240. After image graying and filtering, the STDA based on sea antenna is tested.


## 3.2. Experimental Methods

The control experiment was used in this experiment. In order to verify the STDA based on sea antenna, this paper uses the same group of video sequences for experiments and compares the experimental results of four different algorithms of target detection and tracking.


## 3.3. Experimental Steps

First of all, we use the interframe difference method, optical flow method, background difference method, and sea antenna-based remote ship target monitoring method to process the remote ship target and record the detection time, analyze and compare the difference between them and the original image, and compare the detection time of each method. Then, the collected video is detected and verified, and a certain continuous time period of a day is selected from the collected data for verification. The period from 9:00 a.m. to 2:00 p.m. on December 12, 2020, is selected to test the statistical effect of the detection module. Because the number of ships in the open sea is relatively small, the time interval set here is 1 hour. Finally, in order to make a more accurate evaluation of the ship tracking effect, we need to calculate and analyze the ship tracking effect. At this time, we introduce two evaluation indexes, namely, the speed index and the accuracy index. Through the calculation of these two indexes, we can more comprehensively understand and analyze the performance of the tracking algorithm.


## 4. Algorithm Test Results and Analysis

### 4.1. Image Detection Effect

When the target moves slowly, the gray value between two adjacent frames does not change much, and the time difference detection method is easy to produce void phenomenon and corresponding unclear boundary, which is not conducive to the tracking of the ship in the later stage; in the actual sea scene, the background model cannot adapt to the scene of sea change well due to the influence of weather change, illumination intensity change, and background model change frequently, which leads to the poor detection quality of the algorithm and is affected by the noise that many filters cannot filter. From the experimental results, there are many wave noise interference besides the ship target, which causes more chaotic image, which is also a disadvantage of the average background model method; the optical flow detection method detects the moving ship in the video image and marks the moving direction of the moving ship in the image. It not only detects the ship target at sea but also detects the wave appearing in the picture accordingly. It can be seen that the wave on the sea surface moves like the ship, and the detection effect is not ideal, which makes it difficult to segment the ship later; finally, the detection and segmentation effect of the STDA based on sea antenna is ideal, and the noise interference caused by wave and sky is eliminated effectively. The experimental results are more ideal than the previous three detection algorithms.


### 4.2. Test Time

The detection time of each algorithm is calculated to verify whether the real-time performance of the ship target monitoring method based on sea antenna meets the requirements. The calculation results are shown in Table1.Table 1
Comparison of real-time performance of different vision detection algorithms.
Detection algorithmAverage detection time (millisecond)Time difference method10.12Optical flow detection method14.36Average background difference method12.67Ship target detection method based on sea antenna13.54It can be seen from Figure1 that the average detection time of time difference method is the shortest, which indicates that the processing speed of the algorithm is the fastest. However, combined with the detection effect, the algorithm cannot meet the requirements of ship detection at sea. The algorithm proposed in this paper can accurately detect ships on the sea and can meet the real-time requirements, so the comprehensive performance of the algorithm proposed in this paper is the best, which meets the requirements of ship detection under the complex background of the sea.Figure 1
Comparison of real-time performance of different vision detection algorithms.


### 4.3. Statistical Ship Effect Analysis of Ship Target Detection Algorithm (STDA) Based on Sea Antenna

The real detection rate and error detection rate commonly used in statistics are used to analyze and evaluate the statistical effect of the algorithm on ships. Table2 shows the statistical effect of the number of ships detected on the sea.Table 2
Statistical effect table of marine ship inspection quantity.
Acquisition timeActual number of shipsReal detection quantityUndetected quantityFalse detection quantityTrue detection rateError detection rate9:00-10:0011101090.1%010:00-11:001091190%10%11:00-12:0013121292.3%15.4%12:00-13:0014131092.9%013:00-14:0015141293.3%13.3%As can be seen from Figure2, the average real detection rate of marine ships is 91.72%, and the average error detection rate is 7.74%. It can be seen that the real detection rate and error detection rate are within a reasonable range. The reasons for missing detection are summarized as follows. The main reason is that the STDA detection range based on sea antenna is close to the sea antenna, while the target not offshore antenna cannot be detected, and there is a certain missing detection rate. Error detection refers to the fact that there may be floating objects near the sea antenna. The detection algorithm detects them as a ship, resulting in error detection, but the error detection rate is 7.74%, which does not affect the subsequent work within the range allowed by the system.Figure 2
Correct and error rate of ship inspection quantity at sea.


### 4.4. Accuracy and Real-Time Analysis of Tracking Algorithm for Marine Ships

#### 4.4.1. Real-Time Analysis of Algorithm

The real-time analysis of tracking algorithm is mainly measured according to the time of processing the results of each frame, which shows the efficiency of the algorithm. The frame rate of the three algorithms in different scenes is analyzed and analyzed to evaluate the speed of the algorithm. The average time of each frame processed by the algorithm in the experimental results of traditional mean shift, original TLD, and improved TLD marine multiobjective ship tracking algorithm is calculated as follows. The real-time analysis of the tracking algorithm is mainly measured according to the processing time of each frame, which shows the effectiveness of the algorithm. The frame rates of the three algorithms in different scenes are analyzed to evaluate the speed of the algorithm. In the experimental results of traditional mean shift, original TLD, and improved TLD marine multitarget ship tracking algorithm, the average time per frame processed by the algorithm is calculated as follows.In Table3, comparing the frame rates of the three algorithms under single target and multitarget, it can be concluded that the average shift time per frame under multitarget, the average time consumption per frame of original TLD/MS, and the average time per frame of improving TLD/MS are longer than those of single target, which shows that the effective rate of multitarget tracking algorithm is higher than that of single target. Draw Figure 3 according to the data in the table.Table 3
Tracking efficiency comparison of different algorithms in different marine scenes.
Marine video sceneMean shift average time per frame (ms)Average time consumption per frame of original TLD (ms)Improve the average time per frame of TLD (ms)Single target19.8948.5633.12Multiple target36.7889. 4337.45Figure 3
Tracking efficiency comparison of different algorithms in different marine scenes.Compared with Figure3, in terms of the real-time performance of the algorithm, in the marine single target ship scene, the traditional mean shift single target tracking algorithm has the lowest average time-consuming per frame, which indicates that its algorithm has the fastest processing speed and the highest running efficiency, which is 19.89 ms. The original TLD single target tracking algorithm has the highest average time-consuming per frame. The improved TLD algorithm improves the processing speed of the algorithm and reduces the processing time from 48.56 ms to 33.12 ms. In the marine multitarget ship scene, the improved TLD multitarget ship tracking algorithm takes the least time per frame, which is 37.45 ms, indicating that the algorithm has improved processing efficiency, and the rapidity and real-time performance greatly meet the requirements of ship monitoring.


#### 4.4.2. Algorithm Accuracy Analysis

After calculating the tracking accuracy and center offset of traditional mean shift, original TLD, and improved TLD algorithm, we can get the statistical results in Figure4.Figure 4
Comparison of tracking accuracy of different algorithms in different marine scenes.


## 4.1. Image Detection Effect

When the target moves slowly, the gray value between two adjacent frames does not change much, and the time difference detection method is easy to produce void phenomenon and corresponding unclear boundary, which is not conducive to the tracking of the ship in the later stage; in the actual sea scene, the background model cannot adapt to the scene of sea change well due to the influence of weather change, illumination intensity change, and background model change frequently, which leads to the poor detection quality of the algorithm and is affected by the noise that many filters cannot filter. From the experimental results, there are many wave noise interference besides the ship target, which causes more chaotic image, which is also a disadvantage of the average background model method; the optical flow detection method detects the moving ship in the video image and marks the moving direction of the moving ship in the image. It not only detects the ship target at sea but also detects the wave appearing in the picture accordingly. It can be seen that the wave on the sea surface moves like the ship, and the detection effect is not ideal, which makes it difficult to segment the ship later; finally, the detection and segmentation effect of the STDA based on sea antenna is ideal, and the noise interference caused by wave and sky is eliminated effectively. The experimental results are more ideal than the previous three detection algorithms.


## 4.2. Test Time

The detection time of each algorithm is calculated to verify whether the real-time performance of the ship target monitoring method based on sea antenna meets the requirements. The calculation results are shown in Table1.Table 1
Comparison of real-time performance of different vision detection algorithms.
Detection algorithmAverage detection time (millisecond)Time difference method10.12Optical flow detection method14.36Average background difference method12.67Ship target detection method based on sea antenna13.54It can be seen from Figure1 that the average detection time of time difference method is the shortest, which indicates that the processing speed of the algorithm is the fastest. However, combined with the detection effect, the algorithm cannot meet the requirements of ship detection at sea. The algorithm proposed in this paper can accurately detect ships on the sea and can meet the real-time requirements, so the comprehensive performance of the algorithm proposed in this paper is the best, which meets the requirements of ship detection under the complex background of the sea.Figure 1
Comparison of real-time performance of different vision detection algorithms.


## 4.3. Statistical Ship Effect Analysis of Ship Target Detection Algorithm (STDA) Based on Sea Antenna

The real detection rate and error detection rate commonly used in statistics are used to analyze and evaluate the statistical effect of the algorithm on ships. Table2 shows the statistical effect of the number of ships detected on the sea.Table 2
Statistical effect table of marine ship inspection quantity.
Acquisition timeActual number of shipsReal detection quantityUndetected quantityFalse detection quantityTrue detection rateError detection rate9:00-10:0011101090.1%010:00-11:001091190%10%11:00-12:0013121292.3%15.4%12:00-13:0014131092.9%013:00-14:0015141293.3%13.3%As can be seen from Figure2, the average real detection rate of marine ships is 91.72%, and the average error detection rate is 7.74%. It can be seen that the real detection rate and error detection rate are within a reasonable range. The reasons for missing detection are summarized as follows. The main reason is that the STDA detection range based on sea antenna is close to the sea antenna, while the target not offshore antenna cannot be detected, and there is a certain missing detection rate. Error detection refers to the fact that there may be floating objects near the sea antenna. The detection algorithm detects them as a ship, resulting in error detection, but the error detection rate is 7.74%, which does not affect the subsequent work within the range allowed by the system.Figure 2
Correct and error rate of ship inspection quantity at sea.


## 4.4. Accuracy and Real-Time Analysis of Tracking Algorithm for Marine Ships

### 4.4.1. Real-Time Analysis of Algorithm

The real-time analysis of tracking algorithm is mainly measured according to the time of processing the results of each frame, which shows the efficiency of the algorithm. The frame rate of the three algorithms in different scenes is analyzed and analyzed to evaluate the speed of the algorithm. The average time of each frame processed by the algorithm in the experimental results of traditional mean shift, original TLD, and improved TLD marine multiobjective ship tracking algorithm is calculated as follows. The real-time analysis of the tracking algorithm is mainly measured according to the processing time of each frame, which shows the effectiveness of the algorithm. The frame rates of the three algorithms in different scenes are analyzed to evaluate the speed of the algorithm. In the experimental results of traditional mean shift, original TLD, and improved TLD marine multitarget ship tracking algorithm, the average time per frame processed by the algorithm is calculated as follows.In Table3, comparing the frame rates of the three algorithms under single target and multitarget, it can be concluded that the average shift time per frame under multitarget, the average time consumption per frame of original TLD/MS, and the average time per frame of improving TLD/MS are longer than those of single target, which shows that the effective rate of multitarget tracking algorithm is higher than that of single target. Draw Figure 3 according to the data in the table.Table 3
Tracking efficiency comparison of different algorithms in different marine scenes.
Marine video sceneMean shift average time per frame (ms)Average time consumption per frame of original TLD (ms)Improve the average time per frame of TLD (ms)Single target19.8948.5633.12Multiple target36.7889. 4337.45Figure 3
Tracking efficiency comparison of different algorithms in different marine scenes.Compared with Figure3, in terms of the real-time performance of the algorithm, in the marine single target ship scene, the traditional mean shift single target tracking algorithm has the lowest average time-consuming per frame, which indicates that its algorithm has the fastest processing speed and the highest running efficiency, which is 19.89 ms. The original TLD single target tracking algorithm has the highest average time-consuming per frame. The improved TLD algorithm improves the processing speed of the algorithm and reduces the processing time from 48.56 ms to 33.12 ms. In the marine multitarget ship scene, the improved TLD multitarget ship tracking algorithm takes the least time per frame, which is 37.45 ms, indicating that the algorithm has improved processing efficiency, and the rapidity and real-time performance greatly meet the requirements of ship monitoring.


### 4.4.2. Algorithm Accuracy Analysis

After calculating the tracking accuracy and center offset of traditional mean shift, original TLD, and improved TLD algorithm, we can get the statistical results in Figure4.Figure 4
Comparison of tracking accuracy of different algorithms in different marine scenes.


## 4.4.1. Real-Time Analysis of Algorithm

The real-time analysis of tracking algorithm is mainly measured according to the time of processing the results of each frame, which shows the efficiency of the algorithm. The frame rate of the three algorithms in different scenes is analyzed and analyzed to evaluate the speed of the algorithm. The average time of each frame processed by the algorithm in the experimental results of traditional mean shift, original TLD, and improved TLD marine multiobjective ship tracking algorithm is calculated as follows. The real-time analysis of the tracking algorithm is mainly measured according to the processing time of each frame, which shows the effectiveness of the algorithm. The frame rates of the three algorithms in different scenes are analyzed to evaluate the speed of the algorithm. In the experimental results of traditional mean shift, original TLD, and improved TLD marine multitarget ship tracking algorithm, the average time per frame processed by the algorithm is calculated as follows.In Table3, comparing the frame rates of the three algorithms under single target and multitarget, it can be concluded that the average shift time per frame under multitarget, the average time consumption per frame of original TLD/MS, and the average time per frame of improving TLD/MS are longer than those of single target, which shows that the effective rate of multitarget tracking algorithm is higher than that of single target. Draw Figure 3 according to the data in the table.Table 3
Tracking efficiency comparison of different algorithms in different marine scenes.
Marine video sceneMean shift average time per frame (ms)Average time consumption per frame of original TLD (ms)Improve the average time per frame of TLD (ms)Single target19.8948.5633.12Multiple target36.7889. 4337.45Figure 3
Tracking efficiency comparison of different algorithms in different marine scenes.Compared with Figure3, in terms of the real-time performance of the algorithm, in the marine single target ship scene, the traditional mean shift single target tracking algorithm has the lowest average time-consuming per frame, which indicates that its algorithm has the fastest processing speed and the highest running efficiency, which is 19.89 ms. The original TLD single target tracking algorithm has the highest average time-consuming per frame. The improved TLD algorithm improves the processing speed of the algorithm and reduces the processing time from 48.56 ms to 33.12 ms. In the marine multitarget ship scene, the improved TLD multitarget ship tracking algorithm takes the least time per frame, which is 37.45 ms, indicating that the algorithm has improved processing efficiency, and the rapidity and real-time performance greatly meet the requirements of ship monitoring.


## 4.4.2. Algorithm Accuracy Analysis

After calculating the tracking accuracy and center offset of traditional mean shift, original TLD, and improved TLD algorithm, we can get the statistical results in Figure4.Figure 4
Comparison of tracking accuracy of different algorithms in different marine scenes.


## 5. Conclusion

In this paper, multitarget remote ship automatic tracking, with close scene anomaly detection and tracking technology, was designed and developed for marine ship detection and tracking system; improved TLD algorithm takes less time per frame, improving the processing speed of the algorithm, making the speed and real-time performance meet the requirements of ship monitoring. The purpose of this project is basically achieved. However, limited by the research time and personal ability, the system designed in this paper still has many shortcomings, so in the future research work, we will consider the following improvements: the algorithm used in this paper only detects and tracks the ship target near the sea antenna, ignoring the ship information in other positions of the sea antenna, leading to the false detection and false statistics of the corresponding ship. Therefore, one can further consider adding a confidence factor and a grey histogram information for comprehensive detection and tracking.



---

*Source: 9582701-2022-05-17.xml*