document: Discovering informative content blocks from Web documents In this paper , we propose a new approach to discover informative contents from a set of tabular documents ( or Web pages ) of a Web site . Our system , InfoDiscoverer , first partitions a page into several content blocks according to HTML tag ( TABLE ) in a Web page . Based on the occurrence of the features ( terms ) in the set of pages , it calculates entropy value of each feature . According to the entropy value of each feature in a content block , the entropy value of the block is defined . By analyzing the information measure , we propose a method to dynamically select the entropy-threshold that partitions blocks into either informative or redundant . Informative content blocks are distinguished parts of the page , whereas redundant content blocks are common parts . Based on the answer set generated from 13 manually tagged news Web sites with a total of 26,518 Web pages , experiments show that both recall and precision rates are greater than 0.956 . That is , using the approach , informative blocks ( news articles ) of these sites can be automatically separated from semantically redundant contents such as advertisements , banners , navigation panels , news categories , etc. . By adopting InfoDiscoverer as the preprocessor of information retrieval and extraction applications , the retrieval and extracting precision will be increased , and the indexing size and extracting complexity will also be reduced .
doc_bio_tags: O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O