Spores on the cooling off corpse of data science
Tags: Machine Learning, Predictive Analytics, Data Science, Big Data, CRM

A few years ago, data scientism was a fresh shiny hot air balloon that took off due to the aristocratic arrogance of prestigious, predictive analytic vendors who simply milked their cows and watched with glass eyes as the era of big data and cloud computation began.

The burner in the balloon’s basket was fueled by the dropping cost of storing and manipulating massive amounts of data, due to the quality big data tools and sophisticated cloud infrastructure that has become widely available. These were hacked together by resident data scientists using mainly R/Python or other open source tools.

The marketing storm of “data driven business”, “BIG data” and “cloud computation” and the decades long hypnotic education about extraordinary value of predictive analytic established a craving appetite for companies all around the world to have their own burner developed. This craving appetite delivered headlines stating that the sexiest job of the century will be data science. This belief is still quite fashionable, and data scientists need not worry about their job yet. However, there are voices and debates question the longtime future of data science.

I am also one of those who believes the bubble is leaking, however I got to this conclusion in a different way, which I will now unfold…

Once upon a time…

The story goes back about fifteen or twenty years, when the classical vendors of predictive analytic software started spreading their products outside the classical application areas such as banking and insurance. They did this first in terms of slideware. Later, slideware evolved into expensive, yet exceptionally unstable, betas. Nevertheless, in the evangelizations of data mining yield to extensive usage of predictive models in various business areas such as: predictive marketing and CRM, financial services, telecommunication, retail, travel, healthcare and pharmaceuticals.

The applications areas of predictive models and the market of predictive analytics is still growing today and predicted to grow in the remainder of the decade by all major research companies. It is also expected that new application areas will strongly emerge such as predictive maintenance. However, I strongly believe that the landscape of predictive analytics is being revolutionized, and will be marked by an alternative to the era of data scientists, or by the new release of data mining software that still copies the data mining workflows invented twenty years ago for classical applications, as aforementioned.

Today, the most widely used applications of predictive analytics are the different from the classical applications for which the classical workflows were invented.

In the early ages of predictive analytics, when it was used strictly for banking and insurance purposes, there was a very high financial impact in relation to the prediction. A single prediction could earn or lose the company hundreds of thousands of dollars. This fact, unsurprisingly, formed the workflow of the classical data mining applications, as well as focusing on the development and fine tuning of a single model, done by a horde of highly trained mathematicians and statisticians, preferably with a PhD.

This culture of “rocket science” still forms the decisions about predictive analytics today. While the financial impact of predictions in the mass applications areas such as: predictive marketing, CRM, retail or travel, is tiny, most of these companies are still purchasing expensive predictive analytic tools that were built for the classical applications. Most of these companies still hire highly trained data miners to use these products, or struggle to recruit versatile data scientists who can build in-house tools to generate predictions whose financial impact can be measured in fractions of pennies.

I’m pretty much sure that most of the business cases of predictive analytics would fail when they tested.

Back to the spores on the corpse

The key to the future of predictive analytics is the ease of applications and speed of deploy-ability. Interpret-ability of a model, or the performance of an individual model becomes less important, and this produces new challenges to be faced by the traditional data mining workflows and software packages. A majority of the applications of predictive analytics would benefit more using Machine Learning as a Service (MLaaS) instead of owning and licensing a standalone product.

MLaaS products will be accessed through well-defined and fairly standardized APIs that open doors for continuous innovations. The most successful candidates in the future of MLaaS providers will feature the management of large amounts of models through advanced model monitoring capabilities. They will support automatic model development, and by closing the fact-feedback-loop, they will also provide online learning and the models got automatically re-trained when there is a drop in predictive performance.

In the very near future, the aforementioned features of MLaaS products will revolutionize the world of predictive analytics as we know it today. Hundreds of classical applications, products, serves and novel IoT applications will benefit from the adaptation capability or “plug and play intelligence” they provide.