# Web Page Content Block Identification with Extended Block Properties
## 3 Block Identification Method for Block Range and Variation Estimation
### Results of Web Page Division to Content Blocks Test Cases
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & **No. of** & **No. of** & & & & **Boundary Range** & **No. of** & **Siblings** \\
**Web** & **Web** & **Potential** & **Filtered** & **Reduced** & & & **Blocks** & \\
**Page No.** & **Page** & **Content** & **Content** & **Single** & **Max-Min** & **with** & **Main** & **Related** \\  & **Tags** & **Blocks** & **Blocks** & **Length** & **Range** & **Siblings** & **Siblings** & **Siblings** \\ \hline
1 & 521 & 502 & 469 & 238 & 166 & 72 & 114 & 42 & 72 \\
2 & 531 & 518 & 494 & 192 & 88 & 104 & 91 & 24 & 67 \\
3 & 393 & 379 & 305 & 136 & 87 & 49 & 72 & 23 & 49 \\
4 & 24,039 & 23,983 & 5743 & 3458 & 1772 & 2816 & 621 & 226 & 395 \\
5 & 981 & 899 & 511 & 256 & 136 & 242 & 128 & 48 & 80 \\
6 & 1087 & 919 & 718 & 281 & 216 & 239 & 218 & 59 & 159 \\
7 & 826 & 763 & 612 & 285 & 185 & 262 & 148 & 42 & 106 \\
8 & 1523 & 1433 & 975 & 472 & 264 & 444 & 144 & 32 & 112 \\
9 & 111 & 87 & 82 & 45 & 38 & 27 & 21 & 4 & 17 \\
10 & 2474 & 2410 & 2174 & 778 & 425 & 797 & 661 & 188 & 473 \\
11 & 768 & 702 & 364 & 168 & 116 & 137 & 111 & 29 & 82 \\
12 & 4990 & 4375 & 2866 & 1346 & 888 & 1091 & 909 & 251 & 658 \\
13 & 2248 & 2125 & 1344 & 436 & 184 & 557 & 223 & 55 & 168 \\ \hline In total & 40,492 & 39,095 & 16,657 & 8091 & 4565 & 6837 & 3461 & 1023 & 2438 \\ \hline WYT & 16,453 & 15,112 & 10,914 & 4633 & 2793 & 4021 & 2840 & 797 & 2043 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Summary of data for automated web page division to content blocks. The reduction of content blocks was achieved in several steps. We know that some tags have nothing to do with content (head tag with its contents, scripts, styles, etc.). Some of these tags (<base/>, <link/>, <meta/>, <style>) are easy to exclude, by selecting only the content of the body tag. This way, we reduce the total amount of tags from 40,492 to 39,095 (16,453 to 15,112 WYT). This amounts to a 3.5% (8% WYT) reduction. The body content should also be filtered since it usually contains tags that add value to the content. However, they do not store content themselves. For example, script tags are often included in the body tag. By filtering body content for tags that are not used to display content, we reduce the number of content tags from 39,095 to 16,657 (15,112 to 10,914 WYT). This equates to a 57% (28% WYT) reduction. In total, the reduction amounts to 59% (27% WYT) compared to the starting value of 40,492 (16,453 WYT) tags. A more advanced reduction cannot be conducted without those simple tag reductions. Detecting content block boundaries rather than all instances of possible content tags allows us to further reduce content blocks to the mentioned 80% (71% WYT) reduction. From the filtered 16,657 (10,914 WYT) blocks, only 8091 (4633 WYT) were left by applying the proposed web page division to content blocks method. The reduction was achieved by identifying variants of different block boundaries and by leaving just one of the multiple identified sibling segments. By grouping content block boundaries and identifying sibling variants of the block, filtered content blocks were reduced. The analysis of these two methods shows that about 51% (58% WYT) of the tags can be classified into boundary ranges. The boundary block typically groups 5 (3 WYT) tags into one block with min-max boundaries for the block. Another form of content block reduction is the identification of relevant blocks and counting the path of one content block rather than all of them. In the analyzed web pages 3461 (2840 WYT) blocks had a sibling block. In the reduced set of content blocks, 1023 (797 WYT) were selected to represent sibling blocks, while 2438 (2043 WYT) were linked to them but eliminated. According to this, sibling blocks have, on average, three instances, but only one-third of them can be stored to represent the block pattern.