# Robot Floor-Tiling Control Method Based on Finite-State Machine and Visual Measurement in Limited FOV

Authors: Shun Wang; Huixing Zhou; Zhongyue Zhang; Xiaoyu Zheng; Yannan Lv
Journal: Advances in Civil Engineering
 (2021)

---

## Abstract

In recent years, accelerative aging society is meeting the short supply of young and middle-aged labor. Particularly, most young people are reluctant to work in the construction industry, which has caused the labor cost of floor tiling to rise year by year. In addition, floor tiling requires workers to continuously bend over or lean over to work, which greatly jeopardizes the physical health of them. Therefore, advanced technology applied in floor tiling is highly demanded to replace the traditional manual method. On the context, the automatic method of floor tiling may promote the transformation and upgrading of the industry. Although a few robots for floor tiling have been developed, the automation of existing systems is still at a low level. This paper proposes a robot floor-tiling control strategy based on visual measurement feedback and finite-state machine, in which the calculation of tile position information in limited field of vision is obtained by an improved Canny edge detection and a Hough linear transformation. Moreover, an algorithm for complementing tile position information based on visual measurement is proposed, and the quality of tile laying is evaluated online. To evaluate the effect of the proposed control strategy, the experimental verifications are given. The experimental results indicate that the proposed method can complete the automatic floor tiling with high accuracy.


---

## Body

## 1. Introduction

### 1.1. Background and Meaning

With the rapid development of robot technology and sensor technology [1, 2], construction robot technology has made considerable progress for decades [3–5]. Many construction robots are applied to perform specific tasks on construction sites, such as construction fabrication robot [6], drilling robot [7], and marking robot [8]. The usage of robots to replace manual building maintenance and construction in harsh environment will improve the construction industry [9, 10], such as reducing the physical intensity labor, avoiding construction hazards, improving construction efficiency, and ensuring construction quality. Hence, researchers began to pay more attention to the research of construction robot technology [11, 12].At present, in the decoration process, the floor tiling still relies on manual operation with some simple instruments (as shown in Figure1), which is labor intensive and time-consuming. The workers must bend over for a long time to perform the floor tiling, which greatly harms their health [13, 14]. The floor-tiling construction requires the workers to have been qualified with professional skills to ensure construction quality. Due to differences in the proficiency of workers, some prominent problems arise in the floor-tiling construction, such as low efficiency, poor tile flatness, serious empty drum, and waste of construction materials caused by rework. In addition, floor tiling is a highly repetitive construction operation, while the size of the tiles is regular. Therefore, automatic floor tiling by the robot has the potential to replace manual operation, which is in accord with modern development direction of floor tiling.Figure 1
Manual floor-tiling construction.


### 1.2. Related Work

Over the past decade, researchers have been devoting greatly into the research of floor-tiling robots. In 1996, Apostolopoulos et al. [15] proposed a concept of mobile robot to automatically lay tiles. In 2000, Navon [16] developed a floor-tiling robot that realized the location of tiles based on vision, and its efficiency is expected to increase by 2 to 5 times. In 2011, Khan et al. [17] presented a semiautomatic floor-tiling robot, which requires some manual operations to achieve floor tiling. In 2019, Li et al. [18] proposed a vision-based adaptive control method for tile paving, and Liu et al. [19, 20] developed a mobile robot for floor tiling that used laser sensors to measure size and position of tiles. However, in existing related studies, the floor-tiling methods adopt semiautomation or human-machine cooperation, and only some simple visual measurements are carried out. The robustness of the systems is poor. Currently, a simple fact is that the automation and intelligence degree of the robots are low.


#### 1.2.1. Ceramic Tile Position Measurement

The ceramic tile position measurement is a prerequisite for accurate floor-tiling control. The existing research results mainly used laser sensors [17, 19, 20] and vision sensors [16–18] to measure the size and position of tiles. The measurement methods with laser sensors require multiple laser sensors or a single sensor for multipoint measurement. Multipoint measurement will make the measurement process more complex and time-consuming. Furthermore, the movement of the laser will induce new errors. The visual measurement method is simple and position characteristic information of ceramic tiles can be extracted from a single image. In addition, the application of robots in industrial environments provides a reference for the application on construction site. Tlach et al. [21] used a measuring assembly on the robot to achieve position measurement, which required the sensor probe to contact the measured object. As for robot online path optimization, Božek [22] proposed a robot path optimization which is applied to spot welding applications in automotive industry. This method realizes online real-time measurement to provide data support for path optimization.Part of the research has studied ceramic tile measurement in a factory environment. Golkar and Prabuwono [23] proposed an image processing algorithm to measure the length of ceramic tiles. The systems used a canny edge detection to identify the boundary of ceramic tiles and calculate the size of ceramic tiles, in which deviation was less than 2 mm. The function of this system is to distinguish the ceramic tiles of different specifications on the production line, so the dimensional accuracy is low. Hocenski et al. [24] proposed a defect detection method based on edge detection technology and adopted histogram subtraction to solve the problem of threshold definition of the canny edge detector. This method has good adaptability to ceramic tiles with different textures. Emam and Sayyedbarzani [25] used binocular vision combined with two cameras to measure the size deviation of ceramic tiles, in which measurement accuracy could reach 0.06 mm. Golkar et al. [26] used three cameras to measure the different dimensions of ceramic tiles (edge curvature, length and width, edge crack, and thickness) from multiple perspectives and obtained the target from the background by using a histogram subtraction. An attempt is made to find a suitable threshold to improve the system accuracy that made an effect to succeed. This system was used for automatic quality inspection of ceramic tiles in a production line. The ceramic tile measurement above mainly focuses on a single ceramic tile which is only one side or top view of one tile in the field of view, in which feature extraction is relatively simple.On the construction site, Deng et al. [27] used Canny edge detection and Hough transform to detect the boundary of the ceramic tile to realize the monitoring of the construction progress of floor tiling. With the comparison of the edge detection results based on different gradient threshold settings, the suitable edge detector threshold was determined. However, the method adopted a manual method to set the threshold, and the measurement accuracy was low and could not meet the needs of robot floor tiling. Lin and Fang [28] proposed a vision-based quality evaluation system for floor tiling, which received the images of finished tiling and extracted geometric features from the images and then evaluated and determined the construction quality of floor tiling. Navon [16] paid attention to the position of the tiles and the damage of the tiles during the floor-tiling process, but the grayscale values of pixels between 31 and 224 were determined as a mistake through the experience in advance, and the robustness was poor.


#### 1.2.2. Robot Floor-Tiling Control Strategy

At present, there is little research on robot floor-tiling control strategies. Apostolopoulos et al. [15] proposed a concept of the floor-tiling process control method based on position and force feedback, but the controller did not carry out actually. Navon [16] used a semiautomatic floor-tiling control method, which required manual assistance for positioning. Khan et al. and Li et al. [17, 18] mainly discussed the control method of floor tiling with a vision servo system. The experiments showed that the control method had better robustness, but the floor-tiling conditions were relatively simple. Some researchers have also studied the control methods of component installation in other task scenarios. Roveda et al. proposed a control framework [29] that can implement industrial interaction tasks in uncertain work scenarios. They also describe a human-robot cooperation methodology to install heavy and bulky components based on marker-based visual servoing and force control [30]. Therefore, the stable control system for robotic floor tiling needs an in-depth study.The control strategy based on the finite-state machine is to summarize it as a finite state when the system is running, and the state switching is realized by external signal excitation. Therefore, the whole system has good stability. The finite-state machine control method has been applied in many fields such as the centralized control for DC microgrids [31], the integrated navigation and control for robots [32], the fuel cell management strategy [33], and the road automatic driving control [34]. Floor tiling is a repetitive construction process. Each floor tiling can be divided into a finite number of motion states. Applying finite-state machine theory to control floor tiling can simplify the control complexity of the floor-tiling process and improve the robustness of the system.


### 1.3. Innovation of This Article

This paper aims at realizing a stable control method of floor tiling and proposes a floor-tiling control strategy based on the vision measurement in the limited FOV and finite-state machine control strategy. The main contributions of this paper are summarized as follows:(1)
It proposes a fast and robust ceramic tile measurement method based on the vision with a limited FOV. Firstly, an image of the floor-tiling process is obtained in the limited FOV. Next, the edge lines of ceramic tiles are attained through an improved Canny edge detection algorithm and Hough transform algorithm. Finally, a clustering method is proposed to distinguish the edges of ceramic tiles, and the deviation between the desired position and actual position of the tile is calculated.(2)
A floor-tiling control strategy based on finite-state machine theory is proposed. The floor-tiling process is split into finite states, and the important state parameters, especially the deviation from the actual position of the ceramic tile to the desired position of that, drive the finite-state machine to act floor-tiling state transition. The control strategy can systematically summarize the state of the floor-tiling process and improve the robustness of the control system.The rest of this paper is organized as follows. In Section2, the framework of the floor tile robot II (FTR-II) system is briefly described, including software and hardware systems. It states the ceramic tile measurement method used to obtain the position of ceramic tiles in Section 3. After that, a floor-tiling control strategy based on a finite-state machine is proposed in Section 4. In Section 5, the experimental verifications with the floor-tiling robot system based on the proposed method are given. Finally, Section 6 summarizes the conclusions and future work.


## 1.1. Background and Meaning

With the rapid development of robot technology and sensor technology [1, 2], construction robot technology has made considerable progress for decades [3–5]. Many construction robots are applied to perform specific tasks on construction sites, such as construction fabrication robot [6], drilling robot [7], and marking robot [8]. The usage of robots to replace manual building maintenance and construction in harsh environment will improve the construction industry [9, 10], such as reducing the physical intensity labor, avoiding construction hazards, improving construction efficiency, and ensuring construction quality. Hence, researchers began to pay more attention to the research of construction robot technology [11, 12].At present, in the decoration process, the floor tiling still relies on manual operation with some simple instruments (as shown in Figure1), which is labor intensive and time-consuming. The workers must bend over for a long time to perform the floor tiling, which greatly harms their health [13, 14]. The floor-tiling construction requires the workers to have been qualified with professional skills to ensure construction quality. Due to differences in the proficiency of workers, some prominent problems arise in the floor-tiling construction, such as low efficiency, poor tile flatness, serious empty drum, and waste of construction materials caused by rework. In addition, floor tiling is a highly repetitive construction operation, while the size of the tiles is regular. Therefore, automatic floor tiling by the robot has the potential to replace manual operation, which is in accord with modern development direction of floor tiling.Figure 1
Manual floor-tiling construction.


## 1.2. Related Work

Over the past decade, researchers have been devoting greatly into the research of floor-tiling robots. In 1996, Apostolopoulos et al. [15] proposed a concept of mobile robot to automatically lay tiles. In 2000, Navon [16] developed a floor-tiling robot that realized the location of tiles based on vision, and its efficiency is expected to increase by 2 to 5 times. In 2011, Khan et al. [17] presented a semiautomatic floor-tiling robot, which requires some manual operations to achieve floor tiling. In 2019, Li et al. [18] proposed a vision-based adaptive control method for tile paving, and Liu et al. [19, 20] developed a mobile robot for floor tiling that used laser sensors to measure size and position of tiles. However, in existing related studies, the floor-tiling methods adopt semiautomation or human-machine cooperation, and only some simple visual measurements are carried out. The robustness of the systems is poor. Currently, a simple fact is that the automation and intelligence degree of the robots are low.


### 1.2.1. Ceramic Tile Position Measurement

The ceramic tile position measurement is a prerequisite for accurate floor-tiling control. The existing research results mainly used laser sensors [17, 19, 20] and vision sensors [16–18] to measure the size and position of tiles. The measurement methods with laser sensors require multiple laser sensors or a single sensor for multipoint measurement. Multipoint measurement will make the measurement process more complex and time-consuming. Furthermore, the movement of the laser will induce new errors. The visual measurement method is simple and position characteristic information of ceramic tiles can be extracted from a single image. In addition, the application of robots in industrial environments provides a reference for the application on construction site. Tlach et al. [21] used a measuring assembly on the robot to achieve position measurement, which required the sensor probe to contact the measured object. As for robot online path optimization, Božek [22] proposed a robot path optimization which is applied to spot welding applications in automotive industry. This method realizes online real-time measurement to provide data support for path optimization.Part of the research has studied ceramic tile measurement in a factory environment. Golkar and Prabuwono [23] proposed an image processing algorithm to measure the length of ceramic tiles. The systems used a canny edge detection to identify the boundary of ceramic tiles and calculate the size of ceramic tiles, in which deviation was less than 2 mm. The function of this system is to distinguish the ceramic tiles of different specifications on the production line, so the dimensional accuracy is low. Hocenski et al. [24] proposed a defect detection method based on edge detection technology and adopted histogram subtraction to solve the problem of threshold definition of the canny edge detector. This method has good adaptability to ceramic tiles with different textures. Emam and Sayyedbarzani [25] used binocular vision combined with two cameras to measure the size deviation of ceramic tiles, in which measurement accuracy could reach 0.06 mm. Golkar et al. [26] used three cameras to measure the different dimensions of ceramic tiles (edge curvature, length and width, edge crack, and thickness) from multiple perspectives and obtained the target from the background by using a histogram subtraction. An attempt is made to find a suitable threshold to improve the system accuracy that made an effect to succeed. This system was used for automatic quality inspection of ceramic tiles in a production line. The ceramic tile measurement above mainly focuses on a single ceramic tile which is only one side or top view of one tile in the field of view, in which feature extraction is relatively simple.On the construction site, Deng et al. [27] used Canny edge detection and Hough transform to detect the boundary of the ceramic tile to realize the monitoring of the construction progress of floor tiling. With the comparison of the edge detection results based on different gradient threshold settings, the suitable edge detector threshold was determined. However, the method adopted a manual method to set the threshold, and the measurement accuracy was low and could not meet the needs of robot floor tiling. Lin and Fang [28] proposed a vision-based quality evaluation system for floor tiling, which received the images of finished tiling and extracted geometric features from the images and then evaluated and determined the construction quality of floor tiling. Navon [16] paid attention to the position of the tiles and the damage of the tiles during the floor-tiling process, but the grayscale values of pixels between 31 and 224 were determined as a mistake through the experience in advance, and the robustness was poor.


### 1.2.2. Robot Floor-Tiling Control Strategy

At present, there is little research on robot floor-tiling control strategies. Apostolopoulos et al. [15] proposed a concept of the floor-tiling process control method based on position and force feedback, but the controller did not carry out actually. Navon [16] used a semiautomatic floor-tiling control method, which required manual assistance for positioning. Khan et al. and Li et al. [17, 18] mainly discussed the control method of floor tiling with a vision servo system. The experiments showed that the control method had better robustness, but the floor-tiling conditions were relatively simple. Some researchers have also studied the control methods of component installation in other task scenarios. Roveda et al. proposed a control framework [29] that can implement industrial interaction tasks in uncertain work scenarios. They also describe a human-robot cooperation methodology to install heavy and bulky components based on marker-based visual servoing and force control [30]. Therefore, the stable control system for robotic floor tiling needs an in-depth study.The control strategy based on the finite-state machine is to summarize it as a finite state when the system is running, and the state switching is realized by external signal excitation. Therefore, the whole system has good stability. The finite-state machine control method has been applied in many fields such as the centralized control for DC microgrids [31], the integrated navigation and control for robots [32], the fuel cell management strategy [33], and the road automatic driving control [34]. Floor tiling is a repetitive construction process. Each floor tiling can be divided into a finite number of motion states. Applying finite-state machine theory to control floor tiling can simplify the control complexity of the floor-tiling process and improve the robustness of the system.


## 1.2.1. Ceramic Tile Position Measurement

The ceramic tile position measurement is a prerequisite for accurate floor-tiling control. The existing research results mainly used laser sensors [17, 19, 20] and vision sensors [16–18] to measure the size and position of tiles. The measurement methods with laser sensors require multiple laser sensors or a single sensor for multipoint measurement. Multipoint measurement will make the measurement process more complex and time-consuming. Furthermore, the movement of the laser will induce new errors. The visual measurement method is simple and position characteristic information of ceramic tiles can be extracted from a single image. In addition, the application of robots in industrial environments provides a reference for the application on construction site. Tlach et al. [21] used a measuring assembly on the robot to achieve position measurement, which required the sensor probe to contact the measured object. As for robot online path optimization, Božek [22] proposed a robot path optimization which is applied to spot welding applications in automotive industry. This method realizes online real-time measurement to provide data support for path optimization.Part of the research has studied ceramic tile measurement in a factory environment. Golkar and Prabuwono [23] proposed an image processing algorithm to measure the length of ceramic tiles. The systems used a canny edge detection to identify the boundary of ceramic tiles and calculate the size of ceramic tiles, in which deviation was less than 2 mm. The function of this system is to distinguish the ceramic tiles of different specifications on the production line, so the dimensional accuracy is low. Hocenski et al. [24] proposed a defect detection method based on edge detection technology and adopted histogram subtraction to solve the problem of threshold definition of the canny edge detector. This method has good adaptability to ceramic tiles with different textures. Emam and Sayyedbarzani [25] used binocular vision combined with two cameras to measure the size deviation of ceramic tiles, in which measurement accuracy could reach 0.06 mm. Golkar et al. [26] used three cameras to measure the different dimensions of ceramic tiles (edge curvature, length and width, edge crack, and thickness) from multiple perspectives and obtained the target from the background by using a histogram subtraction. An attempt is made to find a suitable threshold to improve the system accuracy that made an effect to succeed. This system was used for automatic quality inspection of ceramic tiles in a production line. The ceramic tile measurement above mainly focuses on a single ceramic tile which is only one side or top view of one tile in the field of view, in which feature extraction is relatively simple.On the construction site, Deng et al. [27] used Canny edge detection and Hough transform to detect the boundary of the ceramic tile to realize the monitoring of the construction progress of floor tiling. With the comparison of the edge detection results based on different gradient threshold settings, the suitable edge detector threshold was determined. However, the method adopted a manual method to set the threshold, and the measurement accuracy was low and could not meet the needs of robot floor tiling. Lin and Fang [28] proposed a vision-based quality evaluation system for floor tiling, which received the images of finished tiling and extracted geometric features from the images and then evaluated and determined the construction quality of floor tiling. Navon [16] paid attention to the position of the tiles and the damage of the tiles during the floor-tiling process, but the grayscale values of pixels between 31 and 224 were determined as a mistake through the experience in advance, and the robustness was poor.


## 1.2.2. Robot Floor-Tiling Control Strategy

At present, there is little research on robot floor-tiling control strategies. Apostolopoulos et al. [15] proposed a concept of the floor-tiling process control method based on position and force feedback, but the controller did not carry out actually. Navon [16] used a semiautomatic floor-tiling control method, which required manual assistance for positioning. Khan et al. and Li et al. [17, 18] mainly discussed the control method of floor tiling with a vision servo system. The experiments showed that the control method had better robustness, but the floor-tiling conditions were relatively simple. Some researchers have also studied the control methods of component installation in other task scenarios. Roveda et al. proposed a control framework [29] that can implement industrial interaction tasks in uncertain work scenarios. They also describe a human-robot cooperation methodology to install heavy and bulky components based on marker-based visual servoing and force control [30]. Therefore, the stable control system for robotic floor tiling needs an in-depth study.The control strategy based on the finite-state machine is to summarize it as a finite state when the system is running, and the state switching is realized by external signal excitation. Therefore, the whole system has good stability. The finite-state machine control method has been applied in many fields such as the centralized control for DC microgrids [31], the integrated navigation and control for robots [32], the fuel cell management strategy [33], and the road automatic driving control [34]. Floor tiling is a repetitive construction process. Each floor tiling can be divided into a finite number of motion states. Applying finite-state machine theory to control floor tiling can simplify the control complexity of the floor-tiling process and improve the robustness of the system.


## 1.3. Innovation of This Article

This paper aims at realizing a stable control method of floor tiling and proposes a floor-tiling control strategy based on the vision measurement in the limited FOV and finite-state machine control strategy. The main contributions of this paper are summarized as follows:(1)
It proposes a fast and robust ceramic tile measurement method based on the vision with a limited FOV. Firstly, an image of the floor-tiling process is obtained in the limited FOV. Next, the edge lines of ceramic tiles are attained through an improved Canny edge detection algorithm and Hough transform algorithm. Finally, a clustering method is proposed to distinguish the edges of ceramic tiles, and the deviation between the desired position and actual position of the tile is calculated.(2)
A floor-tiling control strategy based on finite-state machine theory is proposed. The floor-tiling process is split into finite states, and the important state parameters, especially the deviation from the actual position of the ceramic tile to the desired position of that, drive the finite-state machine to act floor-tiling state transition. The control strategy can systematically summarize the state of the floor-tiling process and improve the robustness of the control system.The rest of this paper is organized as follows. In Section2, the framework of the floor tile robot II (FTR-II) system is briefly described, including software and hardware systems. It states the ceramic tile measurement method used to obtain the position of ceramic tiles in Section 3. After that, a floor-tiling control strategy based on a finite-state machine is proposed in Section 4. In Section 5, the experimental verifications with the floor-tiling robot system based on the proposed method are given. Finally, Section 6 summarizes the conclusions and future work.


## 2. Floor-Tiling Robot II (FTR-II) System

Floor-tiling robot II (FTR-II) system that was used to verify the robotic tile-paving control method mentioned in this article consists of four function layers: user layer, system layer, drive layer, and execution layer, as shown in Figure2.Figure 2
Frame of the FTR-II control system.The user layer is mainly for human-computer interaction. The execution layer implements the subactions of the tile-laying process, and its hardware includes UR10 robot, mobile platform, tile grabbing, and sensors. When the contact force exceeds 10 kg, the robot will immediately stop working and issue a warning, so the safety of FTR-II can be guaranteed. The driver layer is to solve the problem that the system layer cannot directly communicate with the execution layer hardware. The system layer realizes system control and real-time data analysis in the process of tile laying. The hardware and software components of the FTR-II system are shown in Table1.Table 1
System components of FTR-II.
Function moduleDevices/toolDescriptionMain controllerEmbedded industrial computer PCX-9168(i) Control hardware platformUnbuntu20.04 + ROS(ii) Control software platformData processing and evaluationOpencv3.4(i) Convert image to grayscale one(ii) Image edge detection(iii) Image feature extractionVisual studio code 1.52.1for unbuntu20.04(i) Floor position calculation(ii) Floor-tiling quality evaluationManipulatorUR10(i) Floor-tiling position movementCeramic tile grabZK2A15K5HL-08(i) Vacuum generatorZPT32CNJ10-B5-A10(ii) Vacuum suction cupsMobile platformMecanum wheel omnidirectional mobile platform(i) Floor-tiling process movingVertical measurement moduleLaser sensor keyence IL-150(i) Vertical measurement feedbackImage captureSONY IMX214 13M pixel CMOS sensor(i) Digitize image


## 3. Vision-Based Ceramic Tile Position Measurement

### 3.1. Floor Tiling Process Analysis

Figure3 shows the entire process of floor tiling on the site. Firstly, the worker manually lays the ceramic tile no.1 as a benchmark and the rest ceramic tiles are laid fully automatic by the robot in a sequence shown in Figure 3 from no. 2 to no.12. Excluding the floor-tiling process of the ceramic tile 1, the situations of the entire floor tiling are classified into 3 cases:(a)
When the robot is laying the ceramic tiles no. 2 or no.3, the ceramic tile is on the right side of the placed tile No.1. Since there is no ceramic tile on the upper side, only the left ceramic tile no. 1 can be used as a benchmark, as shown in Figure4(a).(b)
When the robot is laying the ceramic tiles no. 4, no.7, or no.10, the ceramic tile to be laid is on the lower side of the laid tiles. Since there is no ceramic tile in the left of the view, only the top ceramic tile may be used as a benchmark, as shown in Figure4(b).(c)
When the robot is laying the other ceramic tiles, different from conditions (a) and (b), the ceramic tiles to be laid are all on the lower-right side of the ceramic tiles that are already laid. Thus, the ceramic tiles on the left and the top side could be used as a benchmark, as shown in Figure4(c).Figure 3
The process of ceramic tile laying.Figure 4
The classification of ceramic tile laying: (a) laying ceramic tile on the right side, (b) laying ceramic tile on the lower side, and (c) laying ceramic tile on the lower-right side.
(a)(b)(c)It should be noted that, as the three cases above, more tiles can be laid.


### 3.2. Obtain Ceramic Tile Image in a Limited FOV

The FTR-II can take an image to distinguish the position of a tile from the adjacent one. Then, the coordinate deviation between the actual position and the desired position is calculated. We hope to obtain an image, clear enough, and extract accurate edge feature information from it. We are supposed to take two main problems into consideration while determining the camera position:(1)
The edge features of the ceramic tiles measured are detected accurately by the edge detection algorithm(2)
It is required to reduce the viewing distance between the camera and the ceramic tile to ensure that the actual physical size of each pixel point mapping can be as small as possibleThe camera is mounted on the robotic arm to ensure that the distance between the camera and the tile is small enough to take a clearer image. However, the position of the robotic arm and the tile being laid is fixed. If the robot takes multiple images to cover the full workspace and merge them to create a single image, the camera should be mounted on another manipulator to realize that the position of the tiles remains unchanged during the process of taking images, which would make the system more complicated. According to the above analysis, this paper adopts a method of obtaining ceramic tile images with a limited field of view. Figure5 shows the measurement method diagram of floor tiling based on the vision. The end effector with four suckers is grabbing a ceramic tile to lay, and a camera is mounted on the robotic arm with a connecting device in order to obtain images in the process of ceramic tile laying. It is important to note that the spatial position relationship between the camera and the laid tile is relatively fixed. Figure 6 shows the images of the ceramic tiles in the limited FOV under the three different conditions.Figure 5
The diagram of floor tiling based on visual measurement.Figure 6
Acquire a tile image in a limited FOV: (a) working condition 1; (b) working condition 2; (c) working condition 3; (d) acquired ceramic tile image.
(a)(b)(c)(d)


### 3.3. Edge Detection of Ceramic Tile Image

The Canny edge detection algorithm [35] was firstly proposed by Canny in 1986 and has been widely used in various computer vision systems [36, 37]. Canny edge detection is not susceptible to noise interference and can detect real weak edges. One advantage is that two different thresholds are selected to obtain edges by excluding weak edges. However, the method of manually setting the high and low thresholds is not highly adaptable to varied environment.This paper proposes an improved Canny edge detection algorithm to extract the edges of the ceramic tiles. Figure7 demonstrates the implementation process of the algorithm, including taking an image, Grayscale image, Gaussian filtering, nonmaximum suppression, Gradient calculation, Otsu adaptive threshold, and edge detection image.Figure 7
The improved canny edge detection algorithm.


#### 3.3.1. Convert Image to Grayscale

Since the image obtained by the camera is a color one, we use the formula0.299R+0.587G+0.114B to convert a color image to a gray-level one.


#### 3.3.2. Gaussian Filtering

The discrete Gaussian convolution kernel matrix is2k+1×2k+1 dimension, which is calculated by formula (1). Therein, σ is the variance and k is the dimension of the kernel matrix. It is noticeable that the choice of Hij will affect the performance of the Canny detector. With increasing size, the detector is less sensitive to noise, but the positioning error of edge detection will also increase slightly. This paper uses a 3 ∗ 3 window for detection, and the pixel to be filtered is e. After Gaussian filtering, the gray value of pixel e is shown in formula (2):(1)Hij=12πσexpi−k+12+j−k+122σ2,1≤i,j≤2k+1,(2)e=H∗Ah11h12h13h21h22h23h31h32h33∗abcdefghi=suma×h11b×h12c×h13d×h21e×h22f×h23g×h31h×h32i×h33,where ∗ is the convolution symbol and sum means the sum of all elements in the matrix.


#### 3.3.3. Gradient Calculation

Gxi,j and Gyi,j are the first-order partial derivatives in the x-direction and y-direction, respectively. The magnitude and direction of the gradient are shown in formulas (3) and (4):(3)Gxx,y=Gx2i,j+Gy2i,j,(4)θi,j=arctanGyi,jGxi,j.


#### 3.3.4. Nonmaxima Suppression (NMS)

Nonmaximum suppression is used to obtain a more accurate image edge, which consists of many single pixels. As shown in Figure8, the gradient direction of the observed pixel is perpendicular to the direction of the image edge. There are two adjacent pixel points not on the edge of the image, but the gradient direction may be calculated by formula (4). The gray value of the observed pixel is compared with the gray values of its adjacency pixels, and the gray value of the pixel nonmaximum is set to 0. Later on, the pixels on the image with nonzero gray values are traversed, and all nonmaximum pixels are eliminated.Figure 8
Nonmaximum suppression in the gradient direction.


#### 3.3.5. Self-Adaptive Threshold Selection Based on Otsu Algorithm

The image processed by the NMS method still contains a lot of noise and false edges. Therefore, the Canny algorithm uses double thresholds to eliminate noise and false edges. For the Canny algorithm, it is very important to choose a reasonable set of thresholds. The traditional Canny algorithm uses manually selected double-threshold values and cannot adapt to various environments where the overall brightness is varied. This paper uses a self-adaptive method based on the Otsu to select the threshold value [38].It is supposed thatQ=0,T−1 is the grayscale range of the image pixels. Taking t as the boundary, the set of pixels is divided into Q0=0,t−1 and Q1=t,T−1, and the probabilities are γ0=∑i=0t−1pi and γ1=∑i=tL−tpi=1−γ0, in the given order. The average gray values of Q0 and Q1 are m0=∑i=0t−1ipi/γ0 and m1=∑i=tT−tipi/γ1, respectively. Hence, the average gray value of Q is calculated as m=γ0m0+γ1m1. The variance between the two classes is expressed in(5)ηt2=γ0m0−m2+γ1m−m12=γ0γ1m0−m12.Calculating the maximum value oft through formula (5), we can obtain the upper threshold value that is marked as t∗.Figure9 reveals the detection results of the traditional Canny edge detection algorithm and the improved Canny edge detection algorithm, where a and b are, respectively, the upper and lower limit thresholds. Comparing Figures 9(a) and 9(b), it is suggested that the detection results of the traditional Canny algorithm vary greatly by manually setting different thresholds. With self-adaptive selecting upper and lower thresholds, the edges detected based on the improved Canny edge detection algorithm illustrated in Figure 9(c) are better than that based on the traditional one.Figure 9
Canny edge detection results under different threshold settings: (a) canny edge detection witha = 50 and b = 100; (b) canny edge detection with a = 80 and b = 150; (c) improved canny edge detection.
(a)(b)(c)The gray values of the image are greatly affected by the light, so the same upper and lower threshold cannot be used to detect edges of the tile image obtained in the construction environment, which has different lighting conditions. The improved Canny detection algorithm has better light intensity adaptability and is taken in this study.


### 3.4. Extract Straight Lines and Recognize Tile Boundaries

In ax−o−y cartesian coordinate system, a straight line is expressed as y=kx+b, and points on the same line have the same b and k values. If the straight line is converted to the b−o−k parameter coordinate system, it should be expressed as b=−xk+y. Points on the same straight line have the same b and k values, so in the parameter coordinate system, they intersect at one point. When the number of parameter straight lines that intersect at the same point reaches the preset value, we think that we have found enough points to form a straight line. This is the principle of Hough transform (HT). However, it is possible that the two edges of the image to detect have the same k and b value.In this paper, we use progressive probabilistic Hough transform (PPHT) to extract tile edges in images [39]. In comparison with Hough transform, the main improvements of the PPHT algorithm are described as follows:(1)
Use randomly selected pixels from the input image instead of traversing all pixels(2)
Set the maximum pixel interval thresholdtreNmin and the minimum straight line length threshold treLmin allowed to form the connecting line segment(3)
Find the longest line segment that is unbroken and does not exceed a given thresholdBy setting the maximum pixel interval allowed to compose the connecting line segment, the PPHT algorithm can effectively distinguish two-tile edges detected as one edge by the HT algorithm. For example, Figure10(b) shows that line 1 detected based on the HT algorithm is composed of two-tile boundaries, which coincide exactly on a straight line. When the PPHT algorithm is used to extract a straight line from the above grayscale image (Figure 10(a)), two straight lines can be acquired accurately (Figure 10(c)).Figure 10
Hough transform for detecting ceramic tile boundaries: (a) improved canny edge detection; (b) HT algorithms; (c) PPHT algorithms.
(a)(b)(c)The tile acquired through the PPHT is expressed in the Cartesian coordinate system asakx+bky=ck. The set of detected points is Ck=xi,yi,0≤i≤nk, where nk is the number of detected points on the line marked as k. Furthermore, the average values of x¯k and y¯k are calculated as(6)x¯k=1nk∑1ixi,y¯k=1nk∑1iyi,0<k≤8,0<i≤nk,where nk is the number of points included in Ck. The recognition strategy of tile boundaries is targeted to sort and compare the values of x¯k and y¯k, which are shown in Table 1 and finally to realize the recognition of 8 tile boundaries.Table2 describes a strategy for recognizing the tile boundaries. The summary of the strategy is as follows:(1)
Sortx¯k.(2)
Pick the minimum two pointsx¯i and x¯j, and then, compare them with the corresponding values of y¯i and y¯j.The smaller one is no. 1 boundary, and the larger one is no. 5 boundary.(3)
Pick the maximum two pointsx¯i and x¯j, and then, compare the corresponding values of y¯i and y¯j. The smaller one is no. 4 boundary and the larger one is no. 8 boundary.(4)
Sorty¯k. The same principle recognizes the tile boundaries 2, 3, 6, and 7.Table 2
The recognition strategy of tile boundaries (condition 3 as an example).
Tile boundary numberkx¯ky¯kTwo minimumTwo maximumCompare valuesTwo minimumTwo maximumCompare values1✓1 < 55✓22 < 3✓3✓66 < 7✓7✓4✓4 < 88✓The result is presented in Figure11. We can get 8 straight lines numbered from 1 to 8.Figure 11
Ceramic tile edge detection and position recognition results.


### 3.5. Position Calculation and Coordinate Transformation

When we are laying ceramic tile, the deviation between the current position and the target position is calculated to drive the end effector of the robot to move. In the two-dimensional rectangular coordinatex−o−y system, movement of the ceramic tile from current position to desired position can be divided into two processes: horizontal translate along vector D1D2⟶=Δx,Δy and rotate θ degree about counterclockwise around D2. D2 is the geometric center of the desired position and D1 is the geometric center of the current position, as shown in Figure 12.Figure 12
The actual position and expected position of the ceramic tile.In the three-dimensional motion space of the robot, the motion matrix of the robot end effector is expressed as(7)Tr=−cosθ−sinθ0Δxsinθcosθ0Δy00100001.


### 3.6. Complement Position Information of Ceramic Tiles

To evaluate the quality of tile laying, we need to wait for all tiles to be laid, and the tiles initially solidify after 24 hours to ensure that the ground has sufficient strength. Afterwards, workers can step on the tiles to check the quality of the tiling. Therefore, this paper proposes a real-time quality evaluation method of laying tile based on the position information of tiles obtained during the tile laying process. However, the ceramic tile visual measurement servo system employs a limited FOV method to obtain images so that it cannot obtain the position information of the entire tiles through a single image. Therefore, only when all tiles are laid, we can obtain the position information of tiles.This requires us to use the position information of partial tiles, which are classified by the three different conditions mentioned in front part, to complement the global information of the tiling. The specific method is as follows.In Figure13, tiles 2 and 3 are complemented by data collected under working condition 1, tiles 4, 7, and 10 are complemented by collecting data under working condition 2, and other tiles are complemented by collecting data under working condition 3.Figure 13
Schematic diagram of the completion method of ceramic tile-paving effect.The use of image complements’ algorithms will introduce an assumption: the base tile isl×l in side length and Δl in size deviation. The international standard BSENISO10545-2:1997 defines the characteristics of various shape and size errors of ceramic tiles, as well as the measurement and inspection methods of related characteristics [40]. The international standard ISO13006 : 2012 specifies the tolerance range of shape and size of various types of ceramic tiles [41].We will analyze the impact of the reference tiles error on the measurement results. The actual size of tile 1 isl+Δl11×l+Δl12, where Δl11 is the x-coordinate deviation and Δl12 is the y-coordinate deviation. When we are laying tile 2, the size is calculated as l+Δl21×l+Δl22, where Δl21 and Δl22 are the vertical position deviation and horizontal position deviation of visual inspection, respectively. However, the actual size of tile 2 is l+Δl21+Δl11×l+Δl22+Δl12. In the same way, the tiles in the first column will introduce a x-coordinate deviation Δl11, and the tiles in the first row will introduce a y-coordinate deviation Δl12. When the tiling effect is evaluated, Δl11 and Δl12 as intermediate variables do not affect the final evaluation of the straightness of the tiles. After the effect is eliminated, we conduct a real-time quality evaluation of tiling, and the algorithm outline is as follows:(1)
Obtain the pixel coordinates of the corner point from the edge of the tile measured based on vision(2)
Use the least square method to fit a straight line(3)
Calculate the distance of each point from points on the fitted straight line


## 3.1. Floor Tiling Process Analysis

Figure3 shows the entire process of floor tiling on the site. Firstly, the worker manually lays the ceramic tile no.1 as a benchmark and the rest ceramic tiles are laid fully automatic by the robot in a sequence shown in Figure 3 from no. 2 to no.12. Excluding the floor-tiling process of the ceramic tile 1, the situations of the entire floor tiling are classified into 3 cases:(a)
When the robot is laying the ceramic tiles no. 2 or no.3, the ceramic tile is on the right side of the placed tile No.1. Since there is no ceramic tile on the upper side, only the left ceramic tile no. 1 can be used as a benchmark, as shown in Figure4(a).(b)
When the robot is laying the ceramic tiles no. 4, no.7, or no.10, the ceramic tile to be laid is on the lower side of the laid tiles. Since there is no ceramic tile in the left of the view, only the top ceramic tile may be used as a benchmark, as shown in Figure4(b).(c)
When the robot is laying the other ceramic tiles, different from conditions (a) and (b), the ceramic tiles to be laid are all on the lower-right side of the ceramic tiles that are already laid. Thus, the ceramic tiles on the left and the top side could be used as a benchmark, as shown in Figure4(c).Figure 3
The process of ceramic tile laying.Figure 4
The classification of ceramic tile laying: (a) laying ceramic tile on the right side, (b) laying ceramic tile on the lower side, and (c) laying ceramic tile on the lower-right side.
(a)(b)(c)It should be noted that, as the three cases above, more tiles can be laid.


## 3.2. Obtain Ceramic Tile Image in a Limited FOV

The FTR-II can take an image to distinguish the position of a tile from the adjacent one. Then, the coordinate deviation between the actual position and the desired position is calculated. We hope to obtain an image, clear enough, and extract accurate edge feature information from it. We are supposed to take two main problems into consideration while determining the camera position:(1)
The edge features of the ceramic tiles measured are detected accurately by the edge detection algorithm(2)
It is required to reduce the viewing distance between the camera and the ceramic tile to ensure that the actual physical size of each pixel point mapping can be as small as possibleThe camera is mounted on the robotic arm to ensure that the distance between the camera and the tile is small enough to take a clearer image. However, the position of the robotic arm and the tile being laid is fixed. If the robot takes multiple images to cover the full workspace and merge them to create a single image, the camera should be mounted on another manipulator to realize that the position of the tiles remains unchanged during the process of taking images, which would make the system more complicated. According to the above analysis, this paper adopts a method of obtaining ceramic tile images with a limited field of view. Figure5 shows the measurement method diagram of floor tiling based on the vision. The end effector with four suckers is grabbing a ceramic tile to lay, and a camera is mounted on the robotic arm with a connecting device in order to obtain images in the process of ceramic tile laying. It is important to note that the spatial position relationship between the camera and the laid tile is relatively fixed. Figure 6 shows the images of the ceramic tiles in the limited FOV under the three different conditions.Figure 5
The diagram of floor tiling based on visual measurement.Figure 6
Acquire a tile image in a limited FOV: (a) working condition 1; (b) working condition 2; (c) working condition 3; (d) acquired ceramic tile image.
(a)(b)(c)(d)


## 3.3. Edge Detection of Ceramic Tile Image

The Canny edge detection algorithm [35] was firstly proposed by Canny in 1986 and has been widely used in various computer vision systems [36, 37]. Canny edge detection is not susceptible to noise interference and can detect real weak edges. One advantage is that two different thresholds are selected to obtain edges by excluding weak edges. However, the method of manually setting the high and low thresholds is not highly adaptable to varied environment.This paper proposes an improved Canny edge detection algorithm to extract the edges of the ceramic tiles. Figure7 demonstrates the implementation process of the algorithm, including taking an image, Grayscale image, Gaussian filtering, nonmaximum suppression, Gradient calculation, Otsu adaptive threshold, and edge detection image.Figure 7
The improved canny edge detection algorithm.


### 3.3.1. Convert Image to Grayscale

Since the image obtained by the camera is a color one, we use the formula0.299R+0.587G+0.114B to convert a color image to a gray-level one.


### 3.3.2. Gaussian Filtering

The discrete Gaussian convolution kernel matrix is2k+1×2k+1 dimension, which is calculated by formula (1). Therein, σ is the variance and k is the dimension of the kernel matrix. It is noticeable that the choice of Hij will affect the performance of the Canny detector. With increasing size, the detector is less sensitive to noise, but the positioning error of edge detection will also increase slightly. This paper uses a 3 ∗ 3 window for detection, and the pixel to be filtered is e. After Gaussian filtering, the gray value of pixel e is shown in formula (2):(1)Hij=12πσexpi−k+12+j−k+122σ2,1≤i,j≤2k+1,(2)e=H∗Ah11h12h13h21h22h23h31h32h33∗abcdefghi=suma×h11b×h12c×h13d×h21e×h22f×h23g×h31h×h32i×h33,where ∗ is the convolution symbol and sum means the sum of all elements in the matrix.


### 3.3.3. Gradient Calculation

Gxi,j and Gyi,j are the first-order partial derivatives in the x-direction and y-direction, respectively. The magnitude and direction of the gradient are shown in formulas (3) and (4):(3)Gxx,y=Gx2i,j+Gy2i,j,(4)θi,j=arctanGyi,jGxi,j.


### 3.3.4. Nonmaxima Suppression (NMS)

Nonmaximum suppression is used to obtain a more accurate image edge, which consists of many single pixels. As shown in Figure8, the gradient direction of the observed pixel is perpendicular to the direction of the image edge. There are two adjacent pixel points not on the edge of the image, but the gradient direction may be calculated by formula (4). The gray value of the observed pixel is compared with the gray values of its adjacency pixels, and the gray value of the pixel nonmaximum is set to 0. Later on, the pixels on the image with nonzero gray values are traversed, and all nonmaximum pixels are eliminated.Figure 8
Nonmaximum suppression in the gradient direction.


### 3.3.5. Self-Adaptive Threshold Selection Based on Otsu Algorithm

The image processed by the NMS method still contains a lot of noise and false edges. Therefore, the Canny algorithm uses double thresholds to eliminate noise and false edges. For the Canny algorithm, it is very important to choose a reasonable set of thresholds. The traditional Canny algorithm uses manually selected double-threshold values and cannot adapt to various environments where the overall brightness is varied. This paper uses a self-adaptive method based on the Otsu to select the threshold value [38].It is supposed thatQ=0,T−1 is the grayscale range of the image pixels. Taking t as the boundary, the set of pixels is divided into Q0=0,t−1 and Q1=t,T−1, and the probabilities are γ0=∑i=0t−1pi and γ1=∑i=tL−tpi=1−γ0, in the given order. The average gray values of Q0 and Q1 are m0=∑i=0t−1ipi/γ0 and m1=∑i=tT−tipi/γ1, respectively. Hence, the average gray value of Q is calculated as m=γ0m0+γ1m1. The variance between the two classes is expressed in(5)ηt2=γ0m0−m2+γ1m−m12=γ0γ1m0−m12.Calculating the maximum value oft through formula (5), we can obtain the upper threshold value that is marked as t∗.Figure9 reveals the detection results of the traditional Canny edge detection algorithm and the improved Canny edge detection algorithm, where a and b are, respectively, the upper and lower limit thresholds. Comparing Figures 9(a) and 9(b), it is suggested that the detection results of the traditional Canny algorithm vary greatly by manually setting different thresholds. With self-adaptive selecting upper and lower thresholds, the edges detected based on the improved Canny edge detection algorithm illustrated in Figure 9(c) are better than that based on the traditional one.Figure 9
Canny edge detection results under different threshold settings: (a) canny edge detection witha = 50 and b = 100; (b) canny edge detection with a = 80 and b = 150; (c) improved canny edge detection.
(a)(b)(c)The gray values of the image are greatly affected by the light, so the same upper and lower threshold cannot be used to detect edges of the tile image obtained in the construction environment, which has different lighting conditions. The improved Canny detection algorithm has better light intensity adaptability and is taken in this study.


## 3.3.1. Convert Image to Grayscale

Since the image obtained by the camera is a color one, we use the formula0.299R+0.587G+0.114B to convert a color image to a gray-level one.


## 3.3.2. Gaussian Filtering

The discrete Gaussian convolution kernel matrix is2k+1×2k+1 dimension, which is calculated by formula (1). Therein, σ is the variance and k is the dimension of the kernel matrix. It is noticeable that the choice of Hij will affect the performance of the Canny detector. With increasing size, the detector is less sensitive to noise, but the positioning error of edge detection will also increase slightly. This paper uses a 3 ∗ 3 window for detection, and the pixel to be filtered is e. After Gaussian filtering, the gray value of pixel e is shown in formula (2):(1)Hij=12πσexpi−k+12+j−k+122σ2,1≤i,j≤2k+1,(2)e=H∗Ah11h12h13h21h22h23h31h32h33∗abcdefghi=suma×h11b×h12c×h13d×h21e×h22f×h23g×h31h×h32i×h33,where ∗ is the convolution symbol and sum means the sum of all elements in the matrix.


## 3.3.3. Gradient Calculation

Gxi,j and Gyi,j are the first-order partial derivatives in the x-direction and y-direction, respectively. The magnitude and direction of the gradient are shown in formulas (3) and (4):(3)Gxx,y=Gx2i,j+Gy2i,j,(4)θi,j=arctanGyi,jGxi,j.


## 3.3.4. Nonmaxima Suppression (NMS)

Nonmaximum suppression is used to obtain a more accurate image edge, which consists of many single pixels. As shown in Figure8, the gradient direction of the observed pixel is perpendicular to the direction of the image edge. There are two adjacent pixel points not on the edge of the image, but the gradient direction may be calculated by formula (4). The gray value of the observed pixel is compared with the gray values of its adjacency pixels, and the gray value of the pixel nonmaximum is set to 0. Later on, the pixels on the image with nonzero gray values are traversed, and all nonmaximum pixels are eliminated.Figure 8
Nonmaximum suppression in the gradient direction.


## 3.3.5. Self-Adaptive Threshold Selection Based on Otsu Algorithm

The image processed by the NMS method still contains a lot of noise and false edges. Therefore, the Canny algorithm uses double thresholds to eliminate noise and false edges. For the Canny algorithm, it is very important to choose a reasonable set of thresholds. The traditional Canny algorithm uses manually selected double-threshold values and cannot adapt to various environments where the overall brightness is varied. This paper uses a self-adaptive method based on the Otsu to select the threshold value [38].It is supposed thatQ=0,T−1 is the grayscale range of the image pixels. Taking t as the boundary, the set of pixels is divided into Q0=0,t−1 and Q1=t,T−1, and the probabilities are γ0=∑i=0t−1pi and γ1=∑i=tL−tpi=1−γ0, in the given order. The average gray values of Q0 and Q1 are m0=∑i=0t−1ipi/γ0 and m1=∑i=tT−tipi/γ1, respectively. Hence, the average gray value of Q is calculated as m=γ0m0+γ1m1. The variance between the two classes is expressed in(5)ηt2=γ0m0−m2+γ1m−m12=γ0γ1m0−m12.Calculating the maximum value oft through formula (5), we can obtain the upper threshold value that is marked as t∗.Figure9 reveals the detection results of the traditional Canny edge detection algorithm and the improved Canny edge detection algorithm, where a and b are, respectively, the upper and lower limit thresholds. Comparing Figures 9(a) and 9(b), it is suggested that the detection results of the traditional Canny algorithm vary greatly by manually setting different thresholds. With self-adaptive selecting upper and lower thresholds, the edges detected based on the improved Canny edge detection algorithm illustrated in Figure 9(c) are better than that based on the traditional one.Figure 9
Canny edge detection results under different threshold settings: (a) canny edge detection witha = 50 and b = 100; (b) canny edge detection with a = 80 and b = 150; (c) improved canny edge detection.
(a)(b)(c)The gray values of the image are greatly affected by the light, so the same upper and lower threshold cannot be used to detect edges of the tile image obtained in the construction environment, which has different lighting conditions. The improved Canny detection algorithm has better light intensity adaptability and is taken in this study.


## 3.4. Extract Straight Lines and Recognize Tile Boundaries

In ax−o−y cartesian coordinate system, a straight line is expressed as y=kx+b, and points on the same line have the same b and k values. If the straight line is converted to the b−o−k parameter coordinate system, it should be expressed as b=−xk+y. Points on the same straight line have the same b and k values, so in the parameter coordinate system, they intersect at one point. When the number of parameter straight lines that intersect at the same point reaches the preset value, we think that we have found enough points to form a straight line. This is the principle of Hough transform (HT). However, it is possible that the two edges of the image to detect have the same k and b value.In this paper, we use progressive probabilistic Hough transform (PPHT) to extract tile edges in images [39]. In comparison with Hough transform, the main improvements of the PPHT algorithm are described as follows:(1)
Use randomly selected pixels from the input image instead of traversing all pixels(2)
Set the maximum pixel interval thresholdtreNmin and the minimum straight line length threshold treLmin allowed to form the connecting line segment(3)
Find the longest line segment that is unbroken and does not exceed a given thresholdBy setting the maximum pixel interval allowed to compose the connecting line segment, the PPHT algorithm can effectively distinguish two-tile edges detected as one edge by the HT algorithm. For example, Figure10(b) shows that line 1 detected based on the HT algorithm is composed of two-tile boundaries, which coincide exactly on a straight line. When the PPHT algorithm is used to extract a straight line from the above grayscale image (Figure 10(a)), two straight lines can be acquired accurately (Figure 10(c)).Figure 10
Hough transform for detecting ceramic tile boundaries: (a) improved canny edge detection; (b) HT algorithms; (c) PPHT algorithms.
(a)(b)(c)The tile acquired through the PPHT is expressed in the Cartesian coordinate system asakx+bky=ck. The set of detected points is Ck=xi,yi,0≤i≤nk, where nk is the number of detected points on the line marked as k. Furthermore, the average values of x¯k and y¯k are calculated as(6)x¯k=1nk∑1ixi,y¯k=1nk∑1iyi,0<k≤8,0<i≤nk,where nk is the number of points included in Ck. The recognition strategy of tile boundaries is targeted to sort and compare the values of x¯k and y¯k, which are shown in Table 1 and finally to realize the recognition of 8 tile boundaries.Table2 describes a strategy for recognizing the tile boundaries. The summary of the strategy is as follows:(1)
Sortx¯k.(2)
Pick the minimum two pointsx¯i and x¯j, and then, compare them with the corresponding values of y¯i and y¯j.The smaller one is no. 1 boundary, and the larger one is no. 5 boundary.(3)
Pick the maximum two pointsx¯i and x¯j, and then, compare the corresponding values of y¯i and y¯j. The smaller one is no. 4 boundary and the larger one is no. 8 boundary.(4)
Sorty¯k. The same principle recognizes the tile boundaries 2, 3, 6, and 7.Table 2
The recognition strategy of tile boundaries (condition 3 as an example).
Tile boundary numberkx¯ky¯kTwo minimumTwo maximumCompare valuesTwo minimumTwo maximumCompare values1✓1 < 55✓22 < 3✓3✓66 < 7✓7✓4✓4 < 88✓The result is presented in Figure11. We can get 8 straight lines numbered from 1 to 8.Figure 11
Ceramic tile edge detection and position recognition results.


## 3.5. Position Calculation and Coordinate Transformation

When we are laying ceramic tile, the deviation between the current position and the target position is calculated to drive the end effector of the robot to move. In the two-dimensional rectangular coordinatex−o−y system, movement of the ceramic tile from current position to desired position can be divided into two processes: horizontal translate along vector D1D2⟶=Δx,Δy and rotate θ degree about counterclockwise around D2. D2 is the geometric center of the desired position and D1 is the geometric center of the current position, as shown in Figure 12.Figure 12
The actual position and expected position of the ceramic tile.In the three-dimensional motion space of the robot, the motion matrix of the robot end effector is expressed as(7)Tr=−cosθ−sinθ0Δxsinθcosθ0Δy00100001.


## 3.6. Complement Position Information of Ceramic Tiles

To evaluate the quality of tile laying, we need to wait for all tiles to be laid, and the tiles initially solidify after 24 hours to ensure that the ground has sufficient strength. Afterwards, workers can step on the tiles to check the quality of the tiling. Therefore, this paper proposes a real-time quality evaluation method of laying tile based on the position information of tiles obtained during the tile laying process. However, the ceramic tile visual measurement servo system employs a limited FOV method to obtain images so that it cannot obtain the position information of the entire tiles through a single image. Therefore, only when all tiles are laid, we can obtain the position information of tiles.This requires us to use the position information of partial tiles, which are classified by the three different conditions mentioned in front part, to complement the global information of the tiling. The specific method is as follows.In Figure13, tiles 2 and 3 are complemented by data collected under working condition 1, tiles 4, 7, and 10 are complemented by collecting data under working condition 2, and other tiles are complemented by collecting data under working condition 3.Figure 13
Schematic diagram of the completion method of ceramic tile-paving effect.The use of image complements’ algorithms will introduce an assumption: the base tile isl×l in side length and Δl in size deviation. The international standard BSENISO10545-2:1997 defines the characteristics of various shape and size errors of ceramic tiles, as well as the measurement and inspection methods of related characteristics [40]. The international standard ISO13006 : 2012 specifies the tolerance range of shape and size of various types of ceramic tiles [41].We will analyze the impact of the reference tiles error on the measurement results. The actual size of tile 1 isl+Δl11×l+Δl12, where Δl11 is the x-coordinate deviation and Δl12 is the y-coordinate deviation. When we are laying tile 2, the size is calculated as l+Δl21×l+Δl22, where Δl21 and Δl22 are the vertical position deviation and horizontal position deviation of visual inspection, respectively. However, the actual size of tile 2 is l+Δl21+Δl11×l+Δl22+Δl12. In the same way, the tiles in the first column will introduce a x-coordinate deviation Δl11, and the tiles in the first row will introduce a y-coordinate deviation Δl12. When the tiling effect is evaluated, Δl11 and Δl12 as intermediate variables do not affect the final evaluation of the straightness of the tiles. After the effect is eliminated, we conduct a real-time quality evaluation of tiling, and the algorithm outline is as follows:(1)
Obtain the pixel coordinates of the corner point from the edge of the tile measured based on vision(2)
Use the least square method to fit a straight line(3)
Calculate the distance of each point from points on the fitted straight line


## 4. Control Strategy Based on FSM

### 4.1. Floor-Tiling Operation Control Variable and Status Parameter Information

The floor-tiling robot system consists of a vision sensor, a mechanical arm, an end effector, a laser sensor, an edge detector, and a position calculator. In the floor-tiling process, the robot operation control variables correspond to different system execution units, which are shown in Table3.Table 3
Robot operation control variables and function description.
Action executionControl variableFunctionCameraY1Perform a camera action to get a pictureRobotic armY2Perform a movement of the robotic armEnd effectorY3Grab and place a ceramic tile with vacuum suction cupsLaser sensorY4Take vertical height measurementEdge detectorY5Detect the tile edgePosition calculationY6Calculate the deviation between the actual position of the tiles and the expected positionThe operating status of the robot mainly reflects the current working status of the floor-tiling robot and is recorded in the form of status parameters, which mainly include the position information of tiles and operating conditions of tiling. Table4 shows the system operation state parameters.Table 4
Robot operating status parameter information.
Status parameterValueFunctionHorizontal position deviationD1D1=Δx,ΔyAngle position deviationD2D2=θVertical height deviationD3D3=hFloor-tiling condition 1con1The floor-tiling condition shown in Figure4(a)Floor-tiling condition 2con2The floor-tiling condition shown in Figure4(b)Floor-tiling condition 3con3The floor-tiling condition shown in Figure4(c)Working statusM0WorkingM1Unworking


### 4.2. Floor-Tiling Control Based on FSM

A finite-state machine (FSM) is a mathematical model with discrete inputs and outputs. It can record the internal state of historical inputs, determine the control execution ability of the next state and behavior according to the current input, and can effectively describe the life cycle of a system.This paper proposes a floor-tiling control strategy based on FSM for the first time. To facilitate the analysis, the FSM describes the action of laying tile as(8)Σfsm=S,A,P,C,where S=S1,S2,…,Sn is a set of n robot states, A=A1,A2,…,An is a set of n robot program actions, P=P1,P2,…,Pn is a set of n robot state parameters, and C=C1,C2,…,Cn is a set of n state transition conditions.The relationship between the 4 sets above is expressed as follows:(9)Si=Ai,Pi,Ci,i=1,2,…,n.To facilitate the analysis and description of the state transition process, a directed graphG is defined as formula (10) based on the graph theory and formula (9):(10)G≜S,E.The floor-tiling states’ set is abstracted as the vertices of the directed graph, and the state transition condition setC is abstracted as the edges of the directed graph; then, the edge set is defined as follows:(11)E≜E1,E2,…,Er=Si,Sj∣ci,j∈Cij,where ci,j is the condition for the transition from state Si to state Sj, which is abstracted as the right of Si,Sj. When the value of ci,j is 1, the condition is satisfied. When the value is 0, the condition is not satisfied, and the value is −1, which means that the condition does not exist. The r adjacent vertices of Si are expressed as follows:(12)Sia≜Si,1a,Si,2a,…,Si,ra=Sj∣Si,Sj∈E,where Si,ia=Ai,ra,Pi,ra,Ci,ra. The FSM diagram of robot floor tiling is shown in Figure 14.Figure 14
The FSM diagram of robot floor tiling based on visual measurement.The state information of the robot floor tiling is shown in Table5. The process of the proposed control strategy for robotic laying a tile is as follows:(1)
In theS0 state, the system checks the state of the robot and performs initialization operations.(2)
When receiving the signalM1, the system turns to S1 and takes a floor-tiling image.(3)
When the image has been acquired, the system runs the edge detector for edge detection. Under working condition 1, the system turns toS2; under working condition 2, the system turns to S3; under working condition 3, the system turns to S4.(4)
When the edge detection is completed, the system turns toS5 and performs plane position calculation to generate a coordinate transformation matrix Tr composed of the translation and rotation from the current position to the desired position.(5)
When theTr matrix has been calculated, the system turns to S8. The system measures the height of the tile to lay from the ground with a laser sensor.(6)
When the position of the tile, including vertical height and horizontal posture, has been acquired, the system controls the robotic arm to move the tile from the current position to the desired position, and the system sequentially turns toS9, S6, and S7.(7)
The system repeats steps (2)–(5) to calculate the position of the tile. If the position deviation of the tile exceeds the preset value, the system runs step (6).(8)
When the position deviation of the tile is within the preset value range, the system turns toS10, the suction cup is released, and the laying of a tile is completed.Table 5
Status information of robot floor tiling.
StatusFunctionActions and parametersConversion conditionS0InitializeInitializeA0,P0C0,1=M1S1Take imageA1=Y1=1C1,2=con1∩Y1=0C1,3=con1∩Y1=0C1,3=con1∩Y1=0S2Edge detectionA2=Y5=1C2,5=con1∩Y5=0S3A3=Y5=1C3,5=con1∩Y5=0S4A4=Y5=1C4,5=con1∩Y5=0S5Calculate positionA5=Y6=1C5,8=Y6=0S6Move robotic armA6=Y4=0,Y2=1,P6=D1C6,7=D1∩Y2=0S7A7=Y4=0,Y2=1,P7=D2C7,1=D2∩Y6=0S9A9=Y4=0,Y2=1C9,6=Y4=0S8Measure heightA8=Y5=0,Y4=1,P8=D3C8,9=D3∩Y6=0C8,10=End∩M0S10Releases tileA10=Y4=0,Y3=0—With the aforementioned method, the robotic floor tiling is realized.


## 4.1. Floor-Tiling Operation Control Variable and Status Parameter Information

The floor-tiling robot system consists of a vision sensor, a mechanical arm, an end effector, a laser sensor, an edge detector, and a position calculator. In the floor-tiling process, the robot operation control variables correspond to different system execution units, which are shown in Table3.Table 3
Robot operation control variables and function description.
Action executionControl variableFunctionCameraY1Perform a camera action to get a pictureRobotic armY2Perform a movement of the robotic armEnd effectorY3Grab and place a ceramic tile with vacuum suction cupsLaser sensorY4Take vertical height measurementEdge detectorY5Detect the tile edgePosition calculationY6Calculate the deviation between the actual position of the tiles and the expected positionThe operating status of the robot mainly reflects the current working status of the floor-tiling robot and is recorded in the form of status parameters, which mainly include the position information of tiles and operating conditions of tiling. Table4 shows the system operation state parameters.Table 4
Robot operating status parameter information.
Status parameterValueFunctionHorizontal position deviationD1D1=Δx,ΔyAngle position deviationD2D2=θVertical height deviationD3D3=hFloor-tiling condition 1con1The floor-tiling condition shown in Figure4(a)Floor-tiling condition 2con2The floor-tiling condition shown in Figure4(b)Floor-tiling condition 3con3The floor-tiling condition shown in Figure4(c)Working statusM0WorkingM1Unworking


## 4.2. Floor-Tiling Control Based on FSM

A finite-state machine (FSM) is a mathematical model with discrete inputs and outputs. It can record the internal state of historical inputs, determine the control execution ability of the next state and behavior according to the current input, and can effectively describe the life cycle of a system.This paper proposes a floor-tiling control strategy based on FSM for the first time. To facilitate the analysis, the FSM describes the action of laying tile as(8)Σfsm=S,A,P,C,where S=S1,S2,…,Sn is a set of n robot states, A=A1,A2,…,An is a set of n robot program actions, P=P1,P2,…,Pn is a set of n robot state parameters, and C=C1,C2,…,Cn is a set of n state transition conditions.The relationship between the 4 sets above is expressed as follows:(9)Si=Ai,Pi,Ci,i=1,2,…,n.To facilitate the analysis and description of the state transition process, a directed graphG is defined as formula (10) based on the graph theory and formula (9):(10)G≜S,E.The floor-tiling states’ set is abstracted as the vertices of the directed graph, and the state transition condition setC is abstracted as the edges of the directed graph; then, the edge set is defined as follows:(11)E≜E1,E2,…,Er=Si,Sj∣ci,j∈Cij,where ci,j is the condition for the transition from state Si to state Sj, which is abstracted as the right of Si,Sj. When the value of ci,j is 1, the condition is satisfied. When the value is 0, the condition is not satisfied, and the value is −1, which means that the condition does not exist. The r adjacent vertices of Si are expressed as follows:(12)Sia≜Si,1a,Si,2a,…,Si,ra=Sj∣Si,Sj∈E,where Si,ia=Ai,ra,Pi,ra,Ci,ra. The FSM diagram of robot floor tiling is shown in Figure 14.Figure 14
The FSM diagram of robot floor tiling based on visual measurement.The state information of the robot floor tiling is shown in Table5. The process of the proposed control strategy for robotic laying a tile is as follows:(1)
In theS0 state, the system checks the state of the robot and performs initialization operations.(2)
When receiving the signalM1, the system turns to S1 and takes a floor-tiling image.(3)
When the image has been acquired, the system runs the edge detector for edge detection. Under working condition 1, the system turns toS2; under working condition 2, the system turns to S3; under working condition 3, the system turns to S4.(4)
When the edge detection is completed, the system turns toS5 and performs plane position calculation to generate a coordinate transformation matrix Tr composed of the translation and rotation from the current position to the desired position.(5)
When theTr matrix has been calculated, the system turns to S8. The system measures the height of the tile to lay from the ground with a laser sensor.(6)
When the position of the tile, including vertical height and horizontal posture, has been acquired, the system controls the robotic arm to move the tile from the current position to the desired position, and the system sequentially turns toS9, S6, and S7.(7)
The system repeats steps (2)–(5) to calculate the position of the tile. If the position deviation of the tile exceeds the preset value, the system runs step (6).(8)
When the position deviation of the tile is within the preset value range, the system turns toS10, the suction cup is released, and the laying of a tile is completed.Table 5
Status information of robot floor tiling.
StatusFunctionActions and parametersConversion conditionS0InitializeInitializeA0,P0C0,1=M1S1Take imageA1=Y1=1C1,2=con1∩Y1=0C1,3=con1∩Y1=0C1,3=con1∩Y1=0S2Edge detectionA2=Y5=1C2,5=con1∩Y5=0S3A3=Y5=1C3,5=con1∩Y5=0S4A4=Y5=1C4,5=con1∩Y5=0S5Calculate positionA5=Y6=1C5,8=Y6=0S6Move robotic armA6=Y4=0,Y2=1,P6=D1C6,7=D1∩Y2=0S7A7=Y4=0,Y2=1,P7=D2C7,1=D2∩Y6=0S9A9=Y4=0,Y2=1C9,6=Y4=0S8Measure heightA8=Y5=0,Y4=1,P8=D3C8,9=D3∩Y6=0C8,10=End∩M0S10Releases tileA10=Y4=0,Y3=0—With the aforementioned method, the robotic floor tiling is realized.


## 5. Experimental Verification and Discussion

### 5.1. Experiment Preparation

To evaluate the performance of the proposed control strategy, this paper uses the floor-tiling robot II (FTR-II) to perform experimental verification at a laboratory site. Before the experiment, the sizes of 12 tiles are measured to ensure that they are suitable and avoid undermining the experimental results. Each tile is measured 3 times, and the average value is calculated. As shown in Table6, the average deviation is 0.05 mm, and the maximum deviation is 0.14 mm.Table 6
Dimensional measurements of ceramic tiles for experiment.
Tile sizeNo. 1 (mm)No. 2 (mm)No. 3 (mm)No. 4 (mm)No. 5 (mm)No. 6 (mm)No. 7 (mm)No. 8 (mm)No. 9 (mm)No. 10 (mm)No. 11 (mm)No. 12 (mm)X-coordinate300.03299.99300.14299.97300.05300.05300.07300.03300.05300.06299.94300.08Y-coordinate300.02300.06300.03300.08299.96299.98300.05300.04300.02300.11300.02300.10


### 5.2. Experiment at a Laboratory Site

Figure15 shows the results of the floor-tiling experiment at a laboratory site. FTR-II continuously lays 12 tiles with a size of 300 mm × 300 mm on the ground, and the expected gap between adjacent tiles is 3 mm. The tiling sequence is shown in Figure 3, where tile 1 is used as a benchmark tile. With the limit of the robotic arm working range, the system lays tiles from no. 2 to no. 6 when the robot mobile platform is standstill. Then, the robot mobile platform moves 600 mm back to meet the working distance range of the robotic arm, and the system lays tiles from no. 7 to no. 12. It should be noted that the movement of the mobile platform will cause a deviation of 2 mm. Since the system uses vision for real-time positioning, the deviations above will not affect the quality of laying tiles.Figure 15
Robotic floor-tiling experimentation at a laboratory site.After the paving is completed, a line laser is used to detect the position of the tiles 2–12. The coordinate data of 48 points are obtained in one floor-tiling experiment, each tile in the 12 tiles with 4 fixed points. We have repeated the experiments 3 times, obtained a total of 144 points of coordinates and deviation values, and calculated the position deviationΔx and Δy, which is shown in Table 7.Table 7
Results of the robot floor-tiling experiment.
ParametersAverage deviation (mm)Standard deviation (mm)Maximum deviation (mm)Fraction of deviations < 0.5 mm (%)Δx0.040.040.5498Δy0.110.030.5199Δx2+Δy20.230.020.7492Figure16 illustrates the frequency distributions of Δx and Δy in the x′−o′−y′ coordinate system. The average deviations in the x-direction and y-direction are 0.04 mm and 0.11 mm, respectively, while 98.5% of the measured values have deviations <0.5 mm. Noticeably, there is no abnormal interruption in the whole floor-tiling process. The system takes 385 s to lay all 12 tiles, and the time required for place one ceramic tile is 32.1 s. Thus, we confirm that the control strategy proposed in this study can be used to perform accurate floor-tiling operations.Figure 16
Frequency distributions ofΔx and Δy at a laboratory site: (a) frequency distribution of Δx; (b) frequency distribution of Δy.
(a)(b)


### 5.3. Position Deviation Based on the Online Evaluation

We get a total of 48 corner coordinates in 8 rows and 6 columns. The corner coordinates of the tiles in the coordinate system are distributed as shown in Figure17. The black point is the origin coordinate, the blue points are the reference corner, the red points are the measurement corner, and the green points are the tile corner complemented. The coordinates’ deviation of each point is calculated, and the results are shown in Table 8, where Δx is the x-coordinate deviation and Δy is the y-coordinate deviation.Figure 17
Calculation results’ distribution of corner points.Table 8
Results of the online detection evaluation.
ParametersAverage deviation (mm)Standard deviation (mm)Maximum deviation (mm)Fraction of deviations < 0.5 mm (%)Δx0.060.160.6489Δy0.180.140.5494Δx2+Δy20.330.190.8484Table8 shows the deviation results calculated by the complement method. The average deviations in the x-direction and y-direction are 0.06 mm and 0.18 mm, respectively, while 91.5% of the measured values have deviations <0.5 mm.By comparing Table6 with Table 7, the deviation of online evaluation is significantly higher than the deviation of the measurement deviation, which is shown in Table 9. It is supposed that the size of the tile is 300 mm × 300 mm, which allows the deviation of the actual size of the tile to be introduced into the online evaluation system.Table 9
Comparison of online evaluation deviation and measurement deviation.
ParametersMeasurement deviation (mm)Evaluation deviation (mm)Error (%)AverageΔx0.040.0650AverageΔy0.110.1863MaximumΔx0.540.6412MaximumΔy0.510.546


## 5.1. Experiment Preparation

To evaluate the performance of the proposed control strategy, this paper uses the floor-tiling robot II (FTR-II) to perform experimental verification at a laboratory site. Before the experiment, the sizes of 12 tiles are measured to ensure that they are suitable and avoid undermining the experimental results. Each tile is measured 3 times, and the average value is calculated. As shown in Table6, the average deviation is 0.05 mm, and the maximum deviation is 0.14 mm.Table 6
Dimensional measurements of ceramic tiles for experiment.
Tile sizeNo. 1 (mm)No. 2 (mm)No. 3 (mm)No. 4 (mm)No. 5 (mm)No. 6 (mm)No. 7 (mm)No. 8 (mm)No. 9 (mm)No. 10 (mm)No. 11 (mm)No. 12 (mm)X-coordinate300.03299.99300.14299.97300.05300.05300.07300.03300.05300.06299.94300.08Y-coordinate300.02300.06300.03300.08299.96299.98300.05300.04300.02300.11300.02300.10


## 5.2. Experiment at a Laboratory Site

Figure15 shows the results of the floor-tiling experiment at a laboratory site. FTR-II continuously lays 12 tiles with a size of 300 mm × 300 mm on the ground, and the expected gap between adjacent tiles is 3 mm. The tiling sequence is shown in Figure 3, where tile 1 is used as a benchmark tile. With the limit of the robotic arm working range, the system lays tiles from no. 2 to no. 6 when the robot mobile platform is standstill. Then, the robot mobile platform moves 600 mm back to meet the working distance range of the robotic arm, and the system lays tiles from no. 7 to no. 12. It should be noted that the movement of the mobile platform will cause a deviation of 2 mm. Since the system uses vision for real-time positioning, the deviations above will not affect the quality of laying tiles.Figure 15
Robotic floor-tiling experimentation at a laboratory site.After the paving is completed, a line laser is used to detect the position of the tiles 2–12. The coordinate data of 48 points are obtained in one floor-tiling experiment, each tile in the 12 tiles with 4 fixed points. We have repeated the experiments 3 times, obtained a total of 144 points of coordinates and deviation values, and calculated the position deviationΔx and Δy, which is shown in Table 7.Table 7
Results of the robot floor-tiling experiment.
ParametersAverage deviation (mm)Standard deviation (mm)Maximum deviation (mm)Fraction of deviations < 0.5 mm (%)Δx0.040.040.5498Δy0.110.030.5199Δx2+Δy20.230.020.7492Figure16 illustrates the frequency distributions of Δx and Δy in the x′−o′−y′ coordinate system. The average deviations in the x-direction and y-direction are 0.04 mm and 0.11 mm, respectively, while 98.5% of the measured values have deviations <0.5 mm. Noticeably, there is no abnormal interruption in the whole floor-tiling process. The system takes 385 s to lay all 12 tiles, and the time required for place one ceramic tile is 32.1 s. Thus, we confirm that the control strategy proposed in this study can be used to perform accurate floor-tiling operations.Figure 16
Frequency distributions ofΔx and Δy at a laboratory site: (a) frequency distribution of Δx; (b) frequency distribution of Δy.
(a)(b)


## 5.3. Position Deviation Based on the Online Evaluation

We get a total of 48 corner coordinates in 8 rows and 6 columns. The corner coordinates of the tiles in the coordinate system are distributed as shown in Figure17. The black point is the origin coordinate, the blue points are the reference corner, the red points are the measurement corner, and the green points are the tile corner complemented. The coordinates’ deviation of each point is calculated, and the results are shown in Table 8, where Δx is the x-coordinate deviation and Δy is the y-coordinate deviation.Figure 17
Calculation results’ distribution of corner points.Table 8
Results of the online detection evaluation.
ParametersAverage deviation (mm)Standard deviation (mm)Maximum deviation (mm)Fraction of deviations < 0.5 mm (%)Δx0.060.160.6489Δy0.180.140.5494Δx2+Δy20.330.190.8484Table8 shows the deviation results calculated by the complement method. The average deviations in the x-direction and y-direction are 0.06 mm and 0.18 mm, respectively, while 91.5% of the measured values have deviations <0.5 mm.By comparing Table6 with Table 7, the deviation of online evaluation is significantly higher than the deviation of the measurement deviation, which is shown in Table 9. It is supposed that the size of the tile is 300 mm × 300 mm, which allows the deviation of the actual size of the tile to be introduced into the online evaluation system.Table 9
Comparison of online evaluation deviation and measurement deviation.
ParametersMeasurement deviation (mm)Evaluation deviation (mm)Error (%)AverageΔx0.040.0650AverageΔy0.110.1863MaximumΔx0.540.6412MaximumΔy0.510.546


## 6. Conclusion and Future Research

This paper presents a robotic floor-tiling control method based on visual measurement feedback and finite-state machine. The self-developed FTR-II floor-tiling robot was used to conduct experiments at a laboratory site. Through the analysis of the experimental results, it can be concluded that(1)
The control method of robot floor tiling proposed in this study is performed successfully(2)
The deviations in thex- and y-coordinates are 0.04 mm and 0.11 mm, respectively, and 98.5% of all deviations are below 0.5 mm(3)
The online evaluation system can realize the real-time online quality evaluation of floor tilingHowever, the main limitation in this study is that the measurement method based on the limited FOV presupposes that the size of the tile is ideal. Dimensional deviation will be accumulated and will cause the tile position to exceed the permitted range. A possible effective solution is that the position information of multiple tiles is obtained at one time by adjusting the height of the camera. In addition, the detector in this paper can only identify the boundary of monochromatic tiles, and the edge detector will be brittle for patterned tiles.The research of this paper may be not only applicable to control robotic floor-tiling scenes but also use for reference in the construction work under similar working conditions, such as curtain wall installation [42], brick masonry [43], and ceiling installation [44, 45]. In order to realize that the floor-tiling robot is able to perform actual tiling operations on real construction sites, more in-depth research direction may be developed on the research results of this paper, such as to solve the navigation and positioning problem of the mobile platform on construction site [46], to monitor the progress of the floor-tiling construction combined with BIM technology [47, 48], and to achieve precise positioning of tile laying with contact constraints through hybrid position/force control.In future research, an edge detector should be developed to adapt to more general lighting conditions and tiles with different patterns. The authors will also develop a hybrid force/position control algorithm based on the visual measurement and force feedback. It may serve for the floor-tiling robot to achieve operation on real construction site and also develop the approaches to plan path and process monitoring on the construction site by the BIM and the mobile platform navigation and positioning technology.



---

*Source: 8372815-2021-09-06.xml*