# A Method for Filtering Pages by Similarity Degree based on Dynamic Programming
## 4 Experimental Analysis
### Comparison with Other Methods  
We compare MFPSDDP with three methods including a Method for Filtering Pages by Containing Strings (MFPCS), a Method for Filtering Pages by Containing Tags and Attributes (MFPCTA), a Method for Filtering Pages by Link Ratio (MFPLR). MFPCS uses string matching to obtain target webpages. MFPCTA traverses the trees of webpages to find a matching tag and the matching attributes of the tag. Link ratio is the ratio of the number of link nodes to the number of all nodes in the tree of a webpage. MFPLR needs to set the most important parameter, which is link ratio. The existing research work mostly set link ratio to 0.25. If the link ratio of a webpage less than 0.25, then we judge the webpage is a detailed information page. The main configurations of four methods are showed in Table 2. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Same Relationship** & _TUF_ & _AF_ & _TAF_ & \(RR\) & \(PR\) & **Similarity Threshold** \\ \hline RPSA & 1000 & 823 & 703 & 70.3\% & 85.4\% & 0.4 \\ CSR & 1000 & 819 & 731 & 73.1\% & 89.3\% & 0.2 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The \(RR\)s and \(PR\)s when we use these test webpages. \begin{table}
\begin{tabular}{c c} \hline \hline
**Filtering Method** & **Main Configuration** \\ \hline \hline MFPCS & The following conditions must be satisfied at the same time. \\
**Condition 1.** Each webpage obtained after filtering must include one of the \\ following strings: “price”, “flash purchase price”, “Jingdong price”, “exclusive \\ price”, or “price spike”. This condition is expressed as follows: \\ _page_.containsString (“price”) or _page_.containsString (“flash purchase price”) **or** \\ _page_.containsString (“Jingdong price”) or _page_.containsString (“exclusive price”) or _page_.containsString (“price spike”) \\ \multirow{2}{*}{MFPCS} & **Condition 2.** Each webpage obtained after filtering must include the string \\ “distribution”. This condition is expressed as follows: \\ _page_.containsString (“distribution”) \\ \multirow{2}{*}{} & **Condition 3.** Each webpage obtained after filtering must include one of the \\  & following strings: “commodity details” or “commodity introduction”. This condition \\ \multirow{2}{*}{} & is expressed as follows: \\ _page_.containsString (“commodity details”) **or** _page_.containsString \\ (“commodity introduction”) \\ \hline \hline MFPCTA & The following condition must be satisfied. \\
**Condition 1.** Each webpage obtained after filtering must include the tag \textless{}div\textgreater{}, \\ and the value of the “class” attribute of the tag \textless{}div\textgreater{} must be “crumb-wrap”. \\ \multirow{2}{*}{} & This condition is expressed as follows: \\ _page_.containsTag (_divTag_) **and** _page_._divTags_.containsAttribute (_classAttribute_) \\ \hline \hline MFPLR & The following condition must be satisfied. \\
**Condition 1.** The link ratio threshold is set to 0.25. This condition is expressed as follows: \\ _page_.linkRatioOut (0.25) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The main configuration of four methods. We compare MFPSDDP with three other methods by \(RR\) and \(PR\). From the experimental results in Figure 5, the RRs and \(PR\)s of MFPSDDP can reach more than 70%. The \(RR\)s of MFPSDDP show a monotonous rising trend. When the number of webpages filtered is nearly 200,000, the \(RR\) of MFPSDDP reaches 92.2%, the \(PR\) of MFPSDDP reaches 85.1%. As the number of webpages filtered increases, the \(RR\)s of MFPCTA are on the rise. The \(RR\) of MFPCTA reaches 96.8% when the number of webpages filtered is nearly 200,000. But the overall \(PR\)s of MFPCTA decrease with the increase of the number of webpages, at a minimum of 72%. The \(RR\)s of MFPLR are relatively stable, which have been over 98%, but the \(PR\)s of MFPLR are relatively low, and the highest \(PR\) of MFPLR is only 32.9%. When the number of webpages filtered is nearly 200,000, the \(RR\)s and \(PR\)s of four filtering methods are showed in Table 3. The \(RR\) of MFPLR is highest, is 98.5%.