The interpretation of a hyper-document is often influenced by its web context. For instance, a university homepage forms the root of an information hierarchy. Statistical approaches by-pass semantics and exploit purely surface orthographical patterns. However, they are in principle limited to the correlation between form and meaning. General purpose NLP techniques suffer from no such in principle limit but do suffer from very real in practice limits.
Linguistic theories are immature, especially where semantics and pragmatics are concerned.
The latter two rely on vast networks of language and commonsense knowledge. Unfortunately, commonsense is a hard AI problem and currently without an extant solution. As a result, NLP approaches currently constitute more pure research programmes than practical technologies. Anecdotal evidence is also available from sources such as the conversation-based Julia project, where the robot's sophisticated discourse competence is the result of cleverly engineered pattern matching. At present it would appear that strong NLP approaches to language understanding are not mature enough to be practical.