Optimized Term Extraction Method Based on Computing Merged Partial CValues Assessing the completeness of a document collection, regarding terminological coverage of a domain of interest, is a complicated task that requires substantial computational resource and human effort. Automated term extraction (ATE) is an important step within this task in our OntoElect approach. It outputs the bags of terms extracted from incrementally enlarged partial document collections for measuring terminological saturation. Saturation is measured iteratively, using our ( thd ) measure of terminological distance between the two bags of terms. The bags of retained significant terms ( T_i ) and ( T_i  1 ) extracted at ith and i  1st iterations are compared ( (thd(T_i ,T_i  1 )) ) until it is detected that ( thd ) went below the individual term significance threshold. The flaw of our conventional approach is that the sequence of input datasets is built by adding an increment of several documents to the previous dataset. Hence, the major part of the documents undergoes term extraction repeatedly, which is counterproductive. In this paper, we propose and prove the validity of the optimized pipeline based on the modified Cvalue method. It processes the disjoint partitions of a collection but not the incrementally enlarged datasets. It computes partial Cvalues and then merges these in the resulting bags of terms. We prove that the results of extraction are statistically the same for the conventional and optimized pipelines. We support this formal result by evaluation experiments to prove document collection and domain independence. By comparing the run times, we prove the efficiency of the optimized pipeline. We also prove experimentally that the optimized pipeline effectively scales up to process document collections of industrial size.

Exploiting MultiDocument Term Extraction to Improve Named Entity Recognition for Major Concept Detection With worldwide events happening everyday, itxe2x80x99s useful to discover the most discussed topics for intelligence and trend analysis. However, existing news media only provide manually annotated tags related to each article. Given huge amount of Web data with new or unknown terms, itxe2x80x99s critical if these topicrelated keywords can be automatically extracted in an efficient way. In this paper, we propose a multidocument term extraction approach to major concept detection. First, major keywords occurring in multiple documents are extracted by statistical term extraction methods. These are then utilized as the dictionary for word segmentation to improve named entity recognition, by which the most relevant concepts in each single document can be extracted. Finally, the extracted concepts are ranked by their importance score based on their popularity and average position in documents. In the experimental results for major online news sources in Chinese, exploiting multidocument term extraction helps to improve the effectiveness of named entity recognition, with the best F1measure of 0.61, a significant 24% improvement over the baseline. By combining the popularity with average position of terms as the importance score, we able to obtain the best precision of 68% when keeping the top25% content in documents. Keyword extraction results for selected major events further validated the effectiveness of our proposed approach using topicrelated news articles. Further investigation is needed to evaluate the performance of the proposed approach with various data sources.