# Table Recognition for Sensitive Data Perception in an IoT Vision Environment
## 3 Recognition Method Design
### Table Restoration and Table-Cell Extraction
#### 3.3.1 Table Restoration by Inverse Perspective Transformation  
The perspective transformation table can be restored easily by inverse perspective transformation using Equations (11) and (12). \[ \begin{split}&[x^{\prime},y^{\prime},w^{\prime}]=[u,v,w]\times A \\ & A=\begin{bmatrix}a_{11}&a_{12}&a_{13}\\ a_{21}&a_{22}&a_{23}\\ a_{31}&a_{32}&a_{33}\end{bmatrix}\end{split}  \tag{11}\]  
\[ \begin{split}& x=\frac{x^{\prime}}{w^{\prime}}=\frac{a_{11}u+a_{ 21}v+a_{31}}{a_{13}u+a_{23}v+a_{33}}\\ & y=\frac{y^{\prime}}{w^{\prime}}=\frac{a_{12}u+a_{22}v+a_{32}}{a_{13 }u+a_{23}v+a_{33}}\end{split}  \tag{12}\]  
In the equations, \(\langle x,y\rangle\) is the coordinate in perspective transformation image, \((u,v)\) is the corresponding coordinate in restoration image, and A is the perspective transformation matrix. From the above equations, it can be seen that the critical problem of inverse perspective transformation is computing a perspective transformation matrix, which requires four points in the perspective transformation table and four corresponding points in the restoration table. The four points selected are the vertexes of the outermost frame of the table, because it can hold most the table information. To get the matrix, we first determine the outermost frame in the perspective transformation table, then compute coordinates of its four vertexes based on the intersection of the edges, and finally compute the corresponding coordinates of those four vertexes in the restoration table. Determining a table's outermost frame is important in computing the matrix. There are still some interference DCCs beyond the table's outermost frame. Therefore, a table's outermost frame-determination algorithm is proposed to remove these interferences and determine the outermost frame. Ruling lines in a table generally are long, so the DCCs, which are shorter than a certain threshold both in the horizontal direction and vertical direction, are first filtered. Furthermore, the outermost frame is a quadrilateral, so its horizontal edges can cover both vertical edges. Based on this feature, as Figure 9 shows, vertical DCCs can be used to determine the range of horizontal DCCs in a table. Correspondingly, horizontal DCCs can be used to determine the range of vertical DCCs in a table. It means that the leftmost x-coordinate of vertical DCCs is the left boundary on horizontal direction of a table, and the rightmost x-coordinate of vertical DCCs is the right boundary on horizontal direction of the table. After that, DCCs that are out of the boundary are filtered. In addition, the leftmost vertical DCC, the rightmost vertical DCC, the topmost horizontal DCC and the bottommost DCC are selected as the outermost frame of the table. Algorithm 1 shows this Table outermost border detection algorithm. Figure 5g shows the outermost frame of the table in Figure 1 by this algorithm. After determining the outermost frame of the table, the intersection points of its edges will be computed as its four vertexes, which can be represented as  
\(V_{transform}=\{(x_{lb},y_{lb}),(x_{rb},y_{rb}),(x_{rt},y_{rt},(x_{lt},y_{lt}))\}\). In addition, given that the length of the longer horizontal edge is \(W_{e}\), and the length of the longer vertical edge is \(H_{e}\), we should restore the table, instead of the whole image, so the corresponding vertexes of \(V_{transform}\) in restoration image is  
\(V_{restoration}=\{(0,0),(W_{e},0),(H_{e},W_{e}),(0,H_{e})\}\). These eight coordinates are put into Equations (11) and (12), then the perspective transformation matrix will be computed, and the table is restored by inverse perspective transformation. In summary, this section demonstrates that a perspective transformation table should be restored by inverse perspective transformation. To implement it, the perspective transformation matrix is computed by obtaining the four vertexes of the table outermost frame. Algorithm 1 shows how to detect table outermost frame. Figure 9: Outermost frame determination.