It would indeed be an interesting situation if the spammers forced the creation of AI (either they doing it to make better spam or someone like Google to deny it).

One of the interesting things I have experienced in my time at Blekko has been that "growth" on the Web isn't really growing all that much. Sure there are trillions of pages being created but there are only so many things the few billion people in and around the Internet care about. There are 'hard information' places, which are things like libraries where reference searches are common, there are 'entity' places, be thay shops or service providers or SOMA startups, and there are "transient" places where information is current and then stale, to be stored and later reconstructed like coral into a 'dead' (in terms of change) but 'useful' base. Seeing the web from the point of view of a web scale crawler and indexer it starts to be clear that the mantra "Organize all the world's information" is getting tantalizingly close to a dynamically stable froth.

I have to believe that Google has figured this out, some of the smartest engineers I've worked with are at Blekko but Google has its share as well. When you trawl through the fishery and all you get are trash fish you start to wonder, "hmm did we actually catch all the fish there are?" So to it goes with "the Web".

I started doing some speculation [1] on how you could value information that was discoverable on the Internet. And one of the schemes I came up with is how many people would find that information "useful", where useful really means they would have some reason of seeking it out. And then scaling that value by the value to them of having it. So for example if my genome was online, there is maybe a dozen people who would find it "useful", and of that dozen probably on the insurance actuaries and perhaps the occasional researcher who would find it "valuable".

Now one takes that unit of applicability/value and scales it again by the "cost" to acquire it (find it, index it, etc). And from that you can compute the total size of the Web. Well estimate it at least. So far my upper bound (based primarily on the fact that world population is stabilizing) is about 72 trillion documents at any given time.

When you look at it that way you can see that ultimately the spammers lose. They lose because over time the actual information that rises to the useful vs cost threshold is identified and classified, or the legitimate channels that provide dynamic information, or the legitimate archival sources that provide distilled information are all, for the most part known and 99.99% of all your users can find everything they want. And as a spammer you are no longer given the free reign of "appear and be indexed" you have to ask for admittance though some form or another. And the level of new credible sources that are created is inherently a function of the number of people that exist, and the number of people that exist is stabilizing.

When Yahoo started with its human curation it was vastly better than anything anyone else could do, and then it was overwhelmed by a combination of growth and algorithms that could much more rapidly infer curation from the social signals of bookmark pages and article reference. Curation has come back into favor, and it combined with machine learning algorithms will create what is essentially a stable corpus of documents[2] known as "the web".

Wikipedia is a great analog for what is happening world wide. All the reference articles they want to put in are nearly done, the number of editors required has gone down not up, and the future of web search is, in my opinion of course, similar. The only new ground on the web is social networking and Google understands that it seems. Its fortunate for them that only deep pocketed entity that is possibly a near time threat there is Microsoft, and since to date Microsoft is trying to do exactly what they did, they benefit from having already been through that part and know exactly what Microsoft will have to do next.

[1] My side hobby is attempting to discern the economics of information.

[2] Documents are just that, pieces of information, I hardly count every page rendered by Angry Birds in a browser as a separate document, in fact applications are them selves a single "document" in the since that some number of people will seek them out, and connect/consume them over what we think of as the "web" interface.