# Rail Fastener Positioning Based on Double Template Matching

Authors: Yijin Qiu; Xingjie Chen; Zhaomin Lv
Journal: Complexity
 (2020)

---

## Abstract

For global template matching (GTM), which is commonly used in the positioning of rail fasteners, only the fastener template is used to search the global image in both two dimensions, which will result in errors in two dimensions, and the lower positioning accuracy will be caused. A positioning method for rail fasteners based on double template matching (DTM) is proposed in this paper, in which the double template contains the rail template and the fastener template. First, the rail template is used to scan the original image in horizontal dimension, and the squared Euclidean distance (SED) is used to obtain the rail positioning in the original image. Combining with the prior knowledge of the fastener template image, the image composed of the rail and the fastener can be obtained, which is called the Rail Area Map (RAM) in this paper. Then, after preprocessing the RAM and the fastener template image, the fastener template image is used to scan the RAM in vertical dimension, and the normalized correlation coefficient (NCC) is used to calculate the similarity between the template and the subgraph of the RAM to achieve precise positioning of the fastener. The proposed DTM method adopts a positioning strategy from coarse to fine, and two templates are used to complete different positioning tasks in their own dimension, respectively. Due to the rail can be precise positioned in horizontal dimension, the error of the fastener positioning in the horizontal dimension can be avoided, and thus, the positioning accuracy can be improved. Experiments on the on-site line fastener images prove that the proposed method can effectively achieve the precise positioning of fasteners.


---

## Body

## 1. Introduction

As one of the most important modern transportation modes, rail transit has been greatly developed all over the world, especially in China, and the total mileage of rail lines has reached the world’s first, besides rail fasteners play a vital role in the safety of trains. According to the research, many train derailment events are caused by the lack of rail fasteners; so, it is necessary to frequently detect rail fasteners [1]. With the rapid growth of China’s railway operating mileage, the problems of high cost, low efficiency, and various safety hazards caused by ordinary manual inspections [2–4] have been unable to meet the needs of frequent inspections of the railway network. Therefore, the realization of automatic detection of railway fasteners [5–7] has become particularly important.In recent years, artificial intelligence technology has made great progress [8–17], and many technologies based on computer vision have been applied into the detection task of railway fasteners [3, 18–21]. However, if the entire image obtained from the line is detected, not only will it fail to meet the real-time requirements of fastener detection due to the long recognition time but also the complex background in the entire railway line image will inevitably cause the detection accuracy decline. In order to improve the detection efficiency and reduce the interference of complex backgrounds, the precisely positioning of rail fasteners has become an indispensable prerequisite for fastener detection [22]. At present, positioning methods based on edge contours [23–25], positioning methods based on pixel statistics (PS) [26], positioning methods based on GTM [27], and improved methods based on the above methods [28] are the main methods which are used in fastener positioning.Feng et al. [29] used the line segment detector (LSD) to determine the edge positions of rails and sleepers by detecting straight lines, and then, the fastener positioning can be realized based on the geometric relationship of sleepers, rails, and fasteners. In the literature [30], the edge density feature is combined with the Random Sample Consensus (RANSAC) to locate the rail and sleeper areas, and then, the positioning of the fastener can also be realized based on the geometric position relationship of the rail, sleeper, and fastener. However, the above two methods are mainly applied to ballasted track, which have sleepers. For ballastless track, in which the sleepers are replaced by monolithic track bed and the fasteners are installed directly on the monolithic track bed, the above methods are not applicable. In addition, ballastless track is gradually replacing the traditional ballasted track.Aiming at the positioning of ballastless track fasteners, Wei et al. [26] used PS to calculate the vertical gray value characteristics of the fastener and horizontal gray value characteristics of the rail, and then, the appropriate thresholds are selected to locate the rail and fastener backing plates, respectively; finally, the location of the fastener area is indirectly located according to the geometric positional relationship between the fastener backing plates and the rail. However, the pixel gray level between the fastener backing plates and the monolithic track bed is very similar, which makes it difficult to determine the threshold for dividing these two areas.In response to the above problems, Yang et al. [31] extract the directional field as the feature descriptor of the fastener, and then, the positioning of the fastener can be achieved by GTM; Fan et al. [32] used the improved local binary pattern algorithm to obtain the fastener target feature and reconstruct the fastener template image and original image, and then, GTM is used to achieve fastener positioning. The GTM can be effectively applied to ballastless track. However, GTM uses a single fastener template to match in the whole image in two dimensions (vertical and horizontal dimensions), so there will be errors in two dimensions, and the fastener positioning accuracy will be reduced.Therefore, a fastener positioning method based on DTM is proposed in this paper. According to prior knowledge, the entire image is vertically penetrated by the rail. Therefore, first, the rail template is used to scan the original image horizontally, and the SED is used to measure the similarity between the rail template and the subgraph of the original image to obtain the rail positioning in the original image. Due to the obvious difference between the gray level of rail and other regions, only one dimension (horizontal direction) is searched, which effectively ensures that there is no horizontal error in rail positioning. According to the width of the fastener template, the RAM including the rail and the fastener can be divided from the original track image, and it can ensure that the width of the RAM image is equal to the width of the fastener template image. Then, the image preprocessing can suppress noise and improve the contrast between the fastener area and other areas, which is beneficial to improve the accuracy of fastener positioning. Finally, the NCC [33–35] is used to measure the similarity between the fastener template image and the subgraph in the RAM to realize the accurate positioning of the fastener. The matching of one dimension on the RAM can avoid errors in the other dimension. Therefore, the proposed DTM can improve the accuracy of fastener positioning.The main innovations of this article are as follows: (1) when the GTM is used to locate the rail fasteners, the only fastener template needs to be scanned and matched in two dimensions of the whole original image at the same time, which will cause errors in both two dimensions. In theory, the DTM method in this paper only has errors in the vertical direction, which improves the accuracy of fastener positioning. (2) Through the experiment of fastener positioning based on DTM, the accuracy of fastener positioning is improved, and the effectiveness of the method is verified.The remainder of this paper is organized as follows. Section2 introduces the theory of GTM. Section 3 describes the theory of the proposed DTM rail fastener positioning method. Section 4 introduces the experimental results of DTM theory, and the improvement of the accuracy of the DTM method compared to the GTM method is verified, besides the accuracy of the DTM method, and the existing common methods have been compared. Section 5 summarizes and prospects the full text.


## 2. Basic Introduction of GTM

The method of GTM is mainly to use the template image to perform the similarity calculations from left to right and top to bottom on the original image, and finally, a correlation coefficient matrix can be obtained: the parameter values in the coefficient matrix are used to compare the similarity between the template image and the subgraph of the original image. The principle is shown in Figure1; the searched image is SW,H, where W is the width of the image, and H is the height of the image; the template is TN,M, where N is the width of the image, and M is the height of the image. The area covered by the template image in the searched image is called the subgraph Sij, i and j are the coordinate positions of the subgraph’s upper left corner in the searched graph, and the search range is 0≤i<W−N, 0≤j<H−M. By using the similarity calculation method, the similarity degree between the template and subgraph is calculated. When the similarity degree between the template graph and subgraph is the maximum, the corresponding subgraph Sij is the matching target.Figure 1
The principle of GTM. (a) Searched imageS S (W, H). (b) Template image T T (N, M).
(a)(b)


## 3. Fastener Positioning Based on DTM

A detailed description of the DTM fastener positioning method proposed in this paper will be provided in this section. The flow chart consists of two modules: template matching in horizontal direction for positioning rail and template matching in vertical direction for positioning fasteners. The comprehensive flow chart is shown in Figure2, and the details are described in the following contents.Figure 2
Flow chart of fastener positioning based on DTM.


### 3.1. Rail Positioning

In order to reduce the amount of calculation, the grayscale processing is used to process the image [36]. Since the color image of the original image is three channels, the data are relatively large, and the calculation is time-consuming. The gray image will be changed from three channels to a single channel, and the data of single channel are relatively small, so it will be easier to process. The equation of the weighted average method is as follows:(1)Vgrayx,y=0.299×VRx,y+0.587×VGx,y+0.114×VBx,y,where Vgrayx,y is the gray value obtained after weighted average, and VRx,y, VGx,y, and VBx,y are the pixel values of the red channel, green channel, and blue channel, respectively.According to the prior knowledge, each original track image contains at least the following three features: (1) the rail runs through the original track image in the vertical direction, the fastener area is horizontally distributed on both sides of the rail in the track image [37], and the size of the fastener area remains the same. (2) The location of the rail is basically fixed in all rail maps, and the horizontal offset is small, that is, the horizontal offset of the rail in the continuously captured images can be ignored. (3) Due to the repeated friction between the wheel and rail, the rail is obviously white and bright, and its average gray value is obviously different from other areas in the image.First, the rail template is made offline where the number of rows in the rail template is equal to the original image, that is,H=M as shown in Figure 3. Then, the template image is used to slide in the horizontal direction in step=1 in the original image, and the similarity value between the template image and the area (subgraph) covered by it will be calculated. According to the prior knowledge, the gray value of the rail is significantly different from other regions. Therefore, the similarity degree between the rail template image and the original image can be measured by using the SED. The calculation formula is shown in the following equation:(2)Ri=∑m=1M∑n=1NTn,m−Sin,m2,where Ri refers to the similarity value between the template image TN,M and the subgraph at coordinates i,0 in track original image S, Tn,m is the gray value at coordinates n,m in the template T, and Sin,m is the gray value at coordinates n,m in subgraph Si. When the template and subgraph are exactly the same, the correlation coefficient is 0. After the original image is completely searched, Rmini is found, and its corresponding subgraph Si is the rail position in the original image.Figure 3
Schematic diagram of template matching in horizontal direction. (a) Searched imageS S (W, H). (b) Template image T T (N, M).
(a)(b)After positioning to the rail, the column coordinates of the RAM containing the fasteners and rail in the original image can be obtained by the following equation:(3)Cl=cl−α,β=wt−α−cr−cl,Cr=cr+β,where Cl is the column coordinate of the left edge of the located RAM in the original image; Cr is the column coordinate of the right edge of the located RAM in the original image; cl is the column coordinate of the left edge of the rail in the original image obtained by template matching in the horizontal direction; cr is the column coordinates of the right edge of the rail in the original image; wt is the width of the fastener template image which is known by prior knowledge; α is the width of the left fastener in the fastener template image which is known by prior knowledge; and β is the parameter that controls the width of the RAM to equal the width of the fastener template image, and it is also the width of the right fastener in the fastener template image.


### 3.2. Fastener Positioning

In this section, the image preprocessing involved in this paper will be first introduced, which includes image enhancement [38] and image denoising [39], and then, the principle of how to use vertical template matching to achieve accurate fastener positioning will be explained.


#### 3.2.1. Image Enhancement Based on Histogram Equalization

The distribution probability of each gray level in the image is shown by the gray histogram of the image. Therefore, histogram equalization method can be used to reconstruct the original image into a new image whose gray value conforms to the balanced distribution by adjusting the gray distribution of an image from nonuniform distribution to uniform distribution. And the histogram equalization has been proved to be effective in track image preprocessing [40]. The pixel value of an image is u≥0, the probability density function is puuk, and its value can be obtained by the following equation:(4)puuk=huk∑k=0L−1huk,k=0,1,…,L−1.The inputu contains L gray levels, and its probability is puuk, k=0,1,…,L−1. puuk is determined by the histogram huk, and huk represents the number of pixels whose gray value is equal to uk. The output vuk has L gray levels, and vuk can be obtained by the following equation:(5)vuk=∑0kpuuk,where vuk is the cumulative distribution probability value of the pixel value uk in the original image, and the rounded guk of the gray value of the new image can be obtained by the following equation:(6)guk=round255×vuk.


#### 3.2.2. Image Denoising Based on the Gaussian Filter

Gaussian filtering can effectively suppress noise and smooth the image. The function expression is as follows:(7)Gx,y,σ=12πσ2e−x2+y2/2σ2.After the gray valuefx,y of the original image at x,y is processed by the Gaussian filter, the gray value of the new image gx,y can be obtained by the following equation:(8)gx,y=fx,y∗Gx,y,σ.


#### 3.2.3. Accurate Positioning of Fasteners

Symmetry is a basic geometric attribute. Image symmetry and feature symmetry are widely used in image processing, and the effect is remarkable [41]. According to the analysis of fastener image characteristics, fastener is a kind of standard axisymmetric object. Therefore, a pair of complete normal fasteners are selected as the fastener template, and according to the control of equation (3), the width of the fastener template is consistent with the width of the RAM; this limits that the RAM can only be scanned from top to bottom by the symmetrical fastener template, and finally, the precise positioning of the fastener will be obtained. The schematic diagram is shown in Figure 4. Due to the complex grayscale characteristics of the fastener and the influence of illumination, if the simple Euclidean distance is used to calculate the similarity here, only the size relationship between pixels will be considered, a larger error will occur, and affect the accuracy. Therefore, considering the accurate positioning of fasteners, the NCC is used to measure the similarity between the template and the rail area map, and the calculation formula is shown in the following equation:(9)Rj=∑m=1M∑n=1NT′n,m×Sj′n,m∑m=1M∑n=1NT′n,m2∑m=1M∑n=1NSj′n,m2,T′=T−1w×h∑m=1M∑n=1NTn,m,Sj′=Sj−1w×h∑m=1M∑n=1NSjn,m,where Rj refers to the similarity value between the fastener template image T and RAM S at coordinates 0,j, Tn,m is the gray value at coordinates n,m in the template T, Sjn,m is the gray value at coordinates n,m in subgraph Sj, and correlation coefficient is 1 for perfect matching, −1 for poor matching, and 0 for no correlation. Find Rmaxj, and its corresponding subimage Sj is the precise positioning of fastener position.Figure 4
3Schematic diagram of template matching in vertical direction. (a) Searched imageS S (W, H). (b) Template image T T (N, M).
(a)(b)According to equation (9), the coordinates of a pair of fasteners can be accurately located, and then, the images of left fastener and right fastener can be obtained by combining the width of the rail template cr−cl, the width of left fastener α, the width of right fastener β, and the height of the fastener template ht. The upper left corner coordinates of the left fastener in the RAM is 0,j, its height is ht, and its width is α. The upper left corner coordinates of the right fastener in the RAM is α+cr−cl,j, its height is ht, and its width is β.


## 3.1. Rail Positioning

In order to reduce the amount of calculation, the grayscale processing is used to process the image [36]. Since the color image of the original image is three channels, the data are relatively large, and the calculation is time-consuming. The gray image will be changed from three channels to a single channel, and the data of single channel are relatively small, so it will be easier to process. The equation of the weighted average method is as follows:(1)Vgrayx,y=0.299×VRx,y+0.587×VGx,y+0.114×VBx,y,where Vgrayx,y is the gray value obtained after weighted average, and VRx,y, VGx,y, and VBx,y are the pixel values of the red channel, green channel, and blue channel, respectively.According to the prior knowledge, each original track image contains at least the following three features: (1) the rail runs through the original track image in the vertical direction, the fastener area is horizontally distributed on both sides of the rail in the track image [37], and the size of the fastener area remains the same. (2) The location of the rail is basically fixed in all rail maps, and the horizontal offset is small, that is, the horizontal offset of the rail in the continuously captured images can be ignored. (3) Due to the repeated friction between the wheel and rail, the rail is obviously white and bright, and its average gray value is obviously different from other areas in the image.First, the rail template is made offline where the number of rows in the rail template is equal to the original image, that is,H=M as shown in Figure 3. Then, the template image is used to slide in the horizontal direction in step=1 in the original image, and the similarity value between the template image and the area (subgraph) covered by it will be calculated. According to the prior knowledge, the gray value of the rail is significantly different from other regions. Therefore, the similarity degree between the rail template image and the original image can be measured by using the SED. The calculation formula is shown in the following equation:(2)Ri=∑m=1M∑n=1NTn,m−Sin,m2,where Ri refers to the similarity value between the template image TN,M and the subgraph at coordinates i,0 in track original image S, Tn,m is the gray value at coordinates n,m in the template T, and Sin,m is the gray value at coordinates n,m in subgraph Si. When the template and subgraph are exactly the same, the correlation coefficient is 0. After the original image is completely searched, Rmini is found, and its corresponding subgraph Si is the rail position in the original image.Figure 3
Schematic diagram of template matching in horizontal direction. (a) Searched imageS S (W, H). (b) Template image T T (N, M).
(a)(b)After positioning to the rail, the column coordinates of the RAM containing the fasteners and rail in the original image can be obtained by the following equation:(3)Cl=cl−α,β=wt−α−cr−cl,Cr=cr+β,where Cl is the column coordinate of the left edge of the located RAM in the original image; Cr is the column coordinate of the right edge of the located RAM in the original image; cl is the column coordinate of the left edge of the rail in the original image obtained by template matching in the horizontal direction; cr is the column coordinates of the right edge of the rail in the original image; wt is the width of the fastener template image which is known by prior knowledge; α is the width of the left fastener in the fastener template image which is known by prior knowledge; and β is the parameter that controls the width of the RAM to equal the width of the fastener template image, and it is also the width of the right fastener in the fastener template image.


## 3.2. Fastener Positioning

In this section, the image preprocessing involved in this paper will be first introduced, which includes image enhancement [38] and image denoising [39], and then, the principle of how to use vertical template matching to achieve accurate fastener positioning will be explained.


### 3.2.1. Image Enhancement Based on Histogram Equalization

The distribution probability of each gray level in the image is shown by the gray histogram of the image. Therefore, histogram equalization method can be used to reconstruct the original image into a new image whose gray value conforms to the balanced distribution by adjusting the gray distribution of an image from nonuniform distribution to uniform distribution. And the histogram equalization has been proved to be effective in track image preprocessing [40]. The pixel value of an image is u≥0, the probability density function is puuk, and its value can be obtained by the following equation:(4)puuk=huk∑k=0L−1huk,k=0,1,…,L−1.The inputu contains L gray levels, and its probability is puuk, k=0,1,…,L−1. puuk is determined by the histogram huk, and huk represents the number of pixels whose gray value is equal to uk. The output vuk has L gray levels, and vuk can be obtained by the following equation:(5)vuk=∑0kpuuk,where vuk is the cumulative distribution probability value of the pixel value uk in the original image, and the rounded guk of the gray value of the new image can be obtained by the following equation:(6)guk=round255×vuk.


### 3.2.2. Image Denoising Based on the Gaussian Filter

Gaussian filtering can effectively suppress noise and smooth the image. The function expression is as follows:(7)Gx,y,σ=12πσ2e−x2+y2/2σ2.After the gray valuefx,y of the original image at x,y is processed by the Gaussian filter, the gray value of the new image gx,y can be obtained by the following equation:(8)gx,y=fx,y∗Gx,y,σ.


### 3.2.3. Accurate Positioning of Fasteners

Symmetry is a basic geometric attribute. Image symmetry and feature symmetry are widely used in image processing, and the effect is remarkable [41]. According to the analysis of fastener image characteristics, fastener is a kind of standard axisymmetric object. Therefore, a pair of complete normal fasteners are selected as the fastener template, and according to the control of equation (3), the width of the fastener template is consistent with the width of the RAM; this limits that the RAM can only be scanned from top to bottom by the symmetrical fastener template, and finally, the precise positioning of the fastener will be obtained. The schematic diagram is shown in Figure 4. Due to the complex grayscale characteristics of the fastener and the influence of illumination, if the simple Euclidean distance is used to calculate the similarity here, only the size relationship between pixels will be considered, a larger error will occur, and affect the accuracy. Therefore, considering the accurate positioning of fasteners, the NCC is used to measure the similarity between the template and the rail area map, and the calculation formula is shown in the following equation:(9)Rj=∑m=1M∑n=1NT′n,m×Sj′n,m∑m=1M∑n=1NT′n,m2∑m=1M∑n=1NSj′n,m2,T′=T−1w×h∑m=1M∑n=1NTn,m,Sj′=Sj−1w×h∑m=1M∑n=1NSjn,m,where Rj refers to the similarity value between the fastener template image T and RAM S at coordinates 0,j, Tn,m is the gray value at coordinates n,m in the template T, Sjn,m is the gray value at coordinates n,m in subgraph Sj, and correlation coefficient is 1 for perfect matching, −1 for poor matching, and 0 for no correlation. Find Rmaxj, and its corresponding subimage Sj is the precise positioning of fastener position.Figure 4
3Schematic diagram of template matching in vertical direction. (a) Searched imageS S (W, H). (b) Template image T T (N, M).
(a)(b)According to equation (9), the coordinates of a pair of fasteners can be accurately located, and then, the images of left fastener and right fastener can be obtained by combining the width of the rail template cr−cl, the width of left fastener α, the width of right fastener β, and the height of the fastener template ht. The upper left corner coordinates of the left fastener in the RAM is 0,j, its height is ht, and its width is α. The upper left corner coordinates of the right fastener in the RAM is α+cr−cl,j, its height is ht, and its width is β.


## 3.2.1. Image Enhancement Based on Histogram Equalization

The distribution probability of each gray level in the image is shown by the gray histogram of the image. Therefore, histogram equalization method can be used to reconstruct the original image into a new image whose gray value conforms to the balanced distribution by adjusting the gray distribution of an image from nonuniform distribution to uniform distribution. And the histogram equalization has been proved to be effective in track image preprocessing [40]. The pixel value of an image is u≥0, the probability density function is puuk, and its value can be obtained by the following equation:(4)puuk=huk∑k=0L−1huk,k=0,1,…,L−1.The inputu contains L gray levels, and its probability is puuk, k=0,1,…,L−1. puuk is determined by the histogram huk, and huk represents the number of pixels whose gray value is equal to uk. The output vuk has L gray levels, and vuk can be obtained by the following equation:(5)vuk=∑0kpuuk,where vuk is the cumulative distribution probability value of the pixel value uk in the original image, and the rounded guk of the gray value of the new image can be obtained by the following equation:(6)guk=round255×vuk.


## 3.2.2. Image Denoising Based on the Gaussian Filter

Gaussian filtering can effectively suppress noise and smooth the image. The function expression is as follows:(7)Gx,y,σ=12πσ2e−x2+y2/2σ2.After the gray valuefx,y of the original image at x,y is processed by the Gaussian filter, the gray value of the new image gx,y can be obtained by the following equation:(8)gx,y=fx,y∗Gx,y,σ.


## 3.2.3. Accurate Positioning of Fasteners

Symmetry is a basic geometric attribute. Image symmetry and feature symmetry are widely used in image processing, and the effect is remarkable [41]. According to the analysis of fastener image characteristics, fastener is a kind of standard axisymmetric object. Therefore, a pair of complete normal fasteners are selected as the fastener template, and according to the control of equation (3), the width of the fastener template is consistent with the width of the RAM; this limits that the RAM can only be scanned from top to bottom by the symmetrical fastener template, and finally, the precise positioning of the fastener will be obtained. The schematic diagram is shown in Figure 4. Due to the complex grayscale characteristics of the fastener and the influence of illumination, if the simple Euclidean distance is used to calculate the similarity here, only the size relationship between pixels will be considered, a larger error will occur, and affect the accuracy. Therefore, considering the accurate positioning of fasteners, the NCC is used to measure the similarity between the template and the rail area map, and the calculation formula is shown in the following equation:(9)Rj=∑m=1M∑n=1NT′n,m×Sj′n,m∑m=1M∑n=1NT′n,m2∑m=1M∑n=1NSj′n,m2,T′=T−1w×h∑m=1M∑n=1NTn,m,Sj′=Sj−1w×h∑m=1M∑n=1NSjn,m,where Rj refers to the similarity value between the fastener template image T and RAM S at coordinates 0,j, Tn,m is the gray value at coordinates n,m in the template T, Sjn,m is the gray value at coordinates n,m in subgraph Sj, and correlation coefficient is 1 for perfect matching, −1 for poor matching, and 0 for no correlation. Find Rmaxj, and its corresponding subimage Sj is the precise positioning of fastener position.Figure 4
3Schematic diagram of template matching in vertical direction. (a) Searched imageS S (W, H). (b) Template image T T (N, M).
(a)(b)According to equation (9), the coordinates of a pair of fasteners can be accurately located, and then, the images of left fastener and right fastener can be obtained by combining the width of the rail template cr−cl, the width of left fastener α, the width of right fastener β, and the height of the fastener template ht. The upper left corner coordinates of the left fastener in the RAM is 0,j, its height is ht, and its width is α. The upper left corner coordinates of the right fastener in the RAM is α+cr−cl,j, its height is ht, and its width is β.


## 4. Experimental Result

In this paper, the real application scenario has been considered, and the resolution of the image captured by the line-scan camera is 2048 × 4096. A total of 631 pieces of fasteners from the Shijiazhuang-Taiyuan railway line were collected. Each image is composed by a pair of left and right fasteners, as shown in Figure5, where in the original image of the ballastless track (i) is the fastener and (ii) is the rail. The templates are shown in Figure 6, where (a) is the rail template, (b) is the symmetrical fastener template, and (c) is the fastener template after image preprocessing.Figure 5
Original track image: (i) fastener and (ii) rail.Figure 6
Template images. (a) The rail template. (b) The symmetrical fastener template. (c) The fastener template after image preprocessing.
(a)(b)(c)


### 4.1. Analysis of Experimental Results

In order to improve the accuracy of fastener positioning, the coarse to the fine positioning method is adopted in this paper. First, the SED is used to measure the similarity between the rail template image and the subgraph of the original track image. The results are shown in Figure7, where the x-axis indicates the column coordinates of the subgraph in the original image, and the y-axis represents the similarity value between the rail template image and each subgraph. As can be seen from the figure, there is a value extremely close to 0 at the column coordinate of about 2200, that is, the most similar area between the rail template image and the original track image. The subgraph determined by this value can realize the positioning of the rail, and then, the column coordinate of the RAM can be calculated by equation (3). It also can be seen that the matching times of the rail are less than 4000 times/frame. The divided RAM shown in Figures 8(a) and 8(b) is the RAM after image preprocessing.Figure 7
Similarity between the rail template and each subpicture.Figure 8
RAM. (a) The RAM segmented from the original image. (b) The RAM after image preprocessing.
(a)(b)After the segmentation of the RAM, the similarity value between the fastener template map and the subgraphs in the RAM can be measured by using the NCC, and the results are shown in Figure9. The row coordinate of the upper edge of the subgraph in the RAM is represented by x-axis, and the similarity value between the fastener template image and the subgraph is represented by y-axis. The experimental result of DTM algorithm in this paper is represented by the red curve, and the experimental result of using the method in this paper but skipping the image preprocessing step is represented by the blue curve. Since the NCC is used to measure the similarity between the template and subgraph in fastener positioning, the closer the similarity value between the fastener template image and the subgraph is 1, the more similar they are. As can be seen from the figure, there is a maximum value 0.871 at the column coordinate of 1008. Comparing the template matching similarity curve before and after image preprocessing, the blue curve has no obvious maximum value, which indicates that the gray feature of fastener is not obvious in the RAM when the image is not preprocessed, but the maximum value of the red curve is obvious, which indicates that the gray feature of fastener is obvious in the RAM when the image is preprocessed. In this experiment, the fastener positioning is realized on the preprocessed image, while the fastener positioning is lost on the non-preprocessed image. At the same time, it can be seen from the figure that the matching times for accurate positioning of fasteners are less than 1400 times/frame.Figure 9
Similarity between the fastener template and each subgraph.The similarity curve of the template matching using the global two dimensions is shown in Figure10, where the number of calculations is represented by x-axis, and the similarity value between the fastener template image and the subgraph of the original image is represented by y-axis. Compared with the DTM method, the number of matches of DTM is less than 5400 times/frame, but the template matching times of the global two dimensions are more than 3.5 million times/frame. The more matching times, the greater the calculation of the method, and the more time will be consumed.Figure 10
Similarity curve of GTM.And as shown in Figure11, the different similarity measurement algorithms are represented by x-axis, the number of fasteners with wrong positioning is represented by y-axis, the number of wrong positioning using DTM algorithm in this paper is counted by the blue histogram, and the number of wrong positioning using the global two-dimensions template matching is counted by the orange histogram. With the optimization of the algorithm, the number of wrong fastener position decreases significantly. Second, the number of wrong fastener position located by using DTM algorithm is significantly less than the number of wrong fastener position located by using the GTM algorithm. Third, when using the SED to measure the similarity, only the distance between the template and the corresponding gray value of the original image is calculated, which will not achieve the precise fastener positioning. However, in the normalized correlation matching [42] and NCC method, after adding the similarity measurement of the change trend and direction of the image content, the number of wrong positioning was significantly reduced.Figure 11
Comparison of the number of wrong positioning between DTM and GTM.The difficulty of precise positioning fasteners is increased in the actual rail line such as broken fasteners, lost fasteners, and fasteners covered by foreign matters. The results of fastener position based on the DTM method in this paper are shown in Figure12, where (a) is the image of loose fastener, (b) is the image of broken fastener, and (c) is the image of fastener obscured by cables. It can be seen that the above interference problems have been effectively solved.Figure 12
Position of the damaged rail fasteners. (a) The image of loose fastener. (b) The image of broken fastener. (c) The image of fastener obscured by cables.
(a)(b)(c)


### 4.2. Evaluation of Experimental Methods

In order to evaluate the performance of this method, the accuracy rate (AR) of positioning is used as the evaluation standard.O is represented as the number of original image s, and S is represented as the number of images successfully segmented.(10)AR=SO×100%.As can be seen from Table1, the accuracy rate of rail positioning is 100%, which proves that in the entire fastener positioning process, the horizontal error is reduced.Table 1
The accuracy rate of rail positioning.
Track line typeO (piece)S (piece)AR (%)Ballastless track631631100As can be seen from Table2, since there is no sleeper in ballastless track, the edge contour method is not compared. And the threshold value of the fastener backing plates in ballastless track is difficult to be determined by the PS method, so the accuracy rate of fastener positioning is only 86.05%. In addition, if the image is not preprocessed, the grayscale features of the fasteners are not obvious in the whole image, which also leads to inaccurate positioning of the fasteners. Therefore, the DTM method in this article is used but without preprocessing the image, and the accuracy of fastener positioning is only 82.09%. Since the matching in global two dimensions, the calculation amount is large, and it will cause errors in both two dimensions; the DTM method is proposed in this paper, and the positioning accuracy of the fastener is significantly improved. The accuracy of DTM is 1.9% higher than GTM, and its accuracy rate is higher than other methods.Table 2
Comparison of positioning accuracy.
Track line typeMethodO (piece)S (piece)AR (%)Ballastless trackPS63154386.05Without image preprocessed DTM63151882.09GTM63159494.14DTM63160696.04


## 4.1. Analysis of Experimental Results

In order to improve the accuracy of fastener positioning, the coarse to the fine positioning method is adopted in this paper. First, the SED is used to measure the similarity between the rail template image and the subgraph of the original track image. The results are shown in Figure7, where the x-axis indicates the column coordinates of the subgraph in the original image, and the y-axis represents the similarity value between the rail template image and each subgraph. As can be seen from the figure, there is a value extremely close to 0 at the column coordinate of about 2200, that is, the most similar area between the rail template image and the original track image. The subgraph determined by this value can realize the positioning of the rail, and then, the column coordinate of the RAM can be calculated by equation (3). It also can be seen that the matching times of the rail are less than 4000 times/frame. The divided RAM shown in Figures 8(a) and 8(b) is the RAM after image preprocessing.Figure 7
Similarity between the rail template and each subpicture.Figure 8
RAM. (a) The RAM segmented from the original image. (b) The RAM after image preprocessing.
(a)(b)After the segmentation of the RAM, the similarity value between the fastener template map and the subgraphs in the RAM can be measured by using the NCC, and the results are shown in Figure9. The row coordinate of the upper edge of the subgraph in the RAM is represented by x-axis, and the similarity value between the fastener template image and the subgraph is represented by y-axis. The experimental result of DTM algorithm in this paper is represented by the red curve, and the experimental result of using the method in this paper but skipping the image preprocessing step is represented by the blue curve. Since the NCC is used to measure the similarity between the template and subgraph in fastener positioning, the closer the similarity value between the fastener template image and the subgraph is 1, the more similar they are. As can be seen from the figure, there is a maximum value 0.871 at the column coordinate of 1008. Comparing the template matching similarity curve before and after image preprocessing, the blue curve has no obvious maximum value, which indicates that the gray feature of fastener is not obvious in the RAM when the image is not preprocessed, but the maximum value of the red curve is obvious, which indicates that the gray feature of fastener is obvious in the RAM when the image is preprocessed. In this experiment, the fastener positioning is realized on the preprocessed image, while the fastener positioning is lost on the non-preprocessed image. At the same time, it can be seen from the figure that the matching times for accurate positioning of fasteners are less than 1400 times/frame.Figure 9
Similarity between the fastener template and each subgraph.The similarity curve of the template matching using the global two dimensions is shown in Figure10, where the number of calculations is represented by x-axis, and the similarity value between the fastener template image and the subgraph of the original image is represented by y-axis. Compared with the DTM method, the number of matches of DTM is less than 5400 times/frame, but the template matching times of the global two dimensions are more than 3.5 million times/frame. The more matching times, the greater the calculation of the method, and the more time will be consumed.Figure 10
Similarity curve of GTM.And as shown in Figure11, the different similarity measurement algorithms are represented by x-axis, the number of fasteners with wrong positioning is represented by y-axis, the number of wrong positioning using DTM algorithm in this paper is counted by the blue histogram, and the number of wrong positioning using the global two-dimensions template matching is counted by the orange histogram. With the optimization of the algorithm, the number of wrong fastener position decreases significantly. Second, the number of wrong fastener position located by using DTM algorithm is significantly less than the number of wrong fastener position located by using the GTM algorithm. Third, when using the SED to measure the similarity, only the distance between the template and the corresponding gray value of the original image is calculated, which will not achieve the precise fastener positioning. However, in the normalized correlation matching [42] and NCC method, after adding the similarity measurement of the change trend and direction of the image content, the number of wrong positioning was significantly reduced.Figure 11
Comparison of the number of wrong positioning between DTM and GTM.The difficulty of precise positioning fasteners is increased in the actual rail line such as broken fasteners, lost fasteners, and fasteners covered by foreign matters. The results of fastener position based on the DTM method in this paper are shown in Figure12, where (a) is the image of loose fastener, (b) is the image of broken fastener, and (c) is the image of fastener obscured by cables. It can be seen that the above interference problems have been effectively solved.Figure 12
Position of the damaged rail fasteners. (a) The image of loose fastener. (b) The image of broken fastener. (c) The image of fastener obscured by cables.
(a)(b)(c)


## 4.2. Evaluation of Experimental Methods

In order to evaluate the performance of this method, the accuracy rate (AR) of positioning is used as the evaluation standard.O is represented as the number of original image s, and S is represented as the number of images successfully segmented.(10)AR=SO×100%.As can be seen from Table1, the accuracy rate of rail positioning is 100%, which proves that in the entire fastener positioning process, the horizontal error is reduced.Table 1
The accuracy rate of rail positioning.
Track line typeO (piece)S (piece)AR (%)Ballastless track631631100As can be seen from Table2, since there is no sleeper in ballastless track, the edge contour method is not compared. And the threshold value of the fastener backing plates in ballastless track is difficult to be determined by the PS method, so the accuracy rate of fastener positioning is only 86.05%. In addition, if the image is not preprocessed, the grayscale features of the fasteners are not obvious in the whole image, which also leads to inaccurate positioning of the fasteners. Therefore, the DTM method in this article is used but without preprocessing the image, and the accuracy of fastener positioning is only 82.09%. Since the matching in global two dimensions, the calculation amount is large, and it will cause errors in both two dimensions; the DTM method is proposed in this paper, and the positioning accuracy of the fastener is significantly improved. The accuracy of DTM is 1.9% higher than GTM, and its accuracy rate is higher than other methods.Table 2
Comparison of positioning accuracy.
Track line typeMethodO (piece)S (piece)AR (%)Ballastless trackPS63154386.05Without image preprocessed DTM63151882.09GTM63159494.14DTM63160696.04


## 5. Conclusion

In this paper, a fastener positioning method based on DTM is proposed to solve the problem of low positioning accuracy caused by GTM scanning in global two directions in the existing fastener positioning methods. First, the rail template is used to scan the whole image in the horizontal direction, and the SED is used to measure the similarity; then, combined with the prior knowledge of the fastener template image, the RAM can be located from the original image, which reduces the complexity of the fastener positioning. Second, through image preprocessing, the image quality is improved, and the gray characteristics of the fastener are more significant. Finally, the symmetric fastener template is used to scan the RAM only in the vertical direction, which reduces the positioning error caused by the two directions in the GTM, and the NCC is used to measure the similarity between the fastener template image and the subgraph in the RAM, through that the precise positioning of the fasteners will be achieved. It has been verified by experiments that this method can be effectively applied to ballastless tracks in practice.When the track fastener positioning is carried out through the original track images, since the original images used in this article only contain a pair of complete fasteners, the method does not discuss how to realize the positioning of multiple fasteners when an original track image contains multiple fasteners. In the future, the team will try to further optimize the proposed method DTM and effectively use the position information, which is obtained by measuring the distance between adjacent fasteners, to realize multiple fastener positioning.



---

*Source: 8316969-2020-10-06.xml*