Providing content-based access to large quantities of text is a difficult task, given our poor understanding of the formal semantics of human language. The most successful approaches to retrieval, routing, and categorization of documents have relied heavily on statistical techniques. We briefly review some of those techniques and point out where better statistical insight could lead to further advances.
Text is the most notable example, though voice, images, and video are of interest as well.
InfoSpiders deal with dimensionality reduction in a localized, situated way. Information retrieval and the statistics of large data sets. NRC Massive Data Sets Workshop, Washington, DC, 1996. Information retrieval and the statistics of large data sets. NRC Massive Data Sets Workshop, Washington, DC, 1996.