## 6 Results  
Of the approximately 2.5 million files in the USPTO corpus, approximately 6% (or 158,290) of them were determined to be irrelevant or unusable. In most cases, these were either HTML or PDF files without full text or duplicate TIFF files. Figure 2 illustrates a sample of the HTML markup for a single patent and the resulting XML markup that was generated for the same HTML data. Aside from the obvious benefit of simplifying the markup in order to make the HTML patents easier to process, one unexpected benefit of the conversion from HTML to XML was that the size of the patent information was significantly reduced. Specifically, this lead to a 14 GB reduction (or 21.2%) in the size of the patent information originally stored in the HTML files. filter uninteresting words and include essential words in a document's dictionary. The precise filtering of the dictionary paired with a _Bag of Words_ model enables the authors to establish a ground truth level of understanding of this corpus' similarity from a pure text mining perspective. Figure 3: The effects of the percent size of the dictionary as the local thresholds change. The Y-axis represents the percent size of the dictionary in logarithmic scale, base 10. Table 5 shows the results for two experiments using the models from Table 2. Both experiments calculated the document-to-document similarity matrices using two instances of the same collection of patent documents: one with HTML tables and one without HTML tables. Experiment 1 examined the similarity matrices for the set of documents without numerical words and Experiment 2 used the exact same set of documents but included numerical words. Table 5 provides statistical evidence that supports our claim that numerical words within patent documents do not yield substantial content for document representation. As summarized in Table 6, minute differences evidenced in the results of Experiments 1 and 2 demonstrate that including numerical words does not improve document-based similarities. Further, the average percent change (in document-based similarities) between the document set with HTML tables and the document set without HTML tables ranged between 4% and 6%. Based on the percent change in similarity, Tables 5 and 6 confirm that the Term Frequency model is unaffected by the inclusion of tabular data or numerical words. \begin{table}
\begin{tabular}{l c} \hline \hline
**Parameter Type** & **Threshold** \\ \hline Local Maximum & \(\infty\) \\ Local Minimum & \(2\) \\ Global Maximum & \(N-1\) \\ Global Minimum & \(2\) \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The parameters selected and implemented for the main experiments; \(N\) is the total number of patent documents parsed. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & Min \% & Max \% & Average \% & Standard \\ Experiment 1 & Change & Change & Change & Deviation \\ \hline Term Frequency & 0.0 & 81.8 & 4.1 & 6.8 \\ Binary IDF & 9.0 \(\times\) 10\({}^{\text{-}5}\) & 74.7 & 5.3 & 7.2 \\ Term Frequency IDF & 5.0 \(\times\) 10\({}^{\text{-}5}\) & 93.7 & 6.1 & 8.3 \\ \hline  & Min \% & Max \% & Average \% & Standard \\ Experiment 2 & Change & Change & Change & Deviation \\ \hline Term Frequency & 0.0 & 81.8 & 4.1 & 6.8 \\ \hline Binary IDF & 2.0 \(\times\) 10\({}^{\text{-}2}\) & 74.9 & 5.5 & 7.6 \\ \hline Term Frequency IDF & 5.0 \(\times\) 10\({}^{\text{-}5}\) & 93.7 & 5.9 & 8.3 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Change in document-to-document similarity matrices when tabular information is excluded from patent documents; Experiment 1 removes numerical words and Experiment 2 includes numerical words. Figure 4 displays two dendrograms created by agglomerative hierarchical clustering of document-to-document similarities using the Binary IDF Model for patent document representation. The top dendrogram illustrates the clustering for patent documents without HTML tables and the bottom dendrogram shows the clustering assignments for the files with HTML tables. In both cases, numerical words were not included.