Words with more than one part speech have multiple tag assignments and are eventually disambiguated by examining the tags of the words in the surrounding context. Once tagging is complete, the concept selection module consults a template to identify index phrases. The selected concepts are normalized by eliminating determiners and pronouns, and the head noun is stemmed. Secondly, the earlier work reported an algorithm which might increase precision by performing a rudimentary degree of phrase clustering. However, both are active areas of research.
Queries are in the narrative-concept format.
All words and phrases are stemmed. Because we are Plan B participants, we are working with the 350M subset of the Wall Street Journal data. We have also performed extensive testing and failure analysis of a 46449-record subset of the TREC training database which consists of all Wall Street Journal articles from 1987. We chose this subset because all of the documents relevant to the training queries for the Plan B participants were drawn from the 1987 subset. The results reported here are from the 1987 subset of the Wall Street Journal database because the results of the full database were not available at the time of this writing.