Your Users Are Telling You a Story, Are You Listening?
By: Josh Sephton
Tags: Big Data, Data Science, Customer Experience

You may have heard more and more about “Big Data” recently; the term has been growing steadily in popularity over the past couple of years. But what exactly is “Big Data”? And how can you use it to your advantage?

We’re creating data quicker today than ever before. 90% of all the data in the world has been created in the last two years.

In the early days of the internet, information was created by individuals and companies for an audience to consume. As Web 2.0 grew, the audience created their own content for sites like Flickr and Facebook. We’ve now reached a time where connected devices are recording data at an unimaginable rate.

A modern car records about 25GB of data per hour from its sensors; information about the road conditions, the driver and the climate to highlight just a few metrics.

This is a huge amount of information. In fact, it’s about the same as 12,500 copies of War and Peace. Just consider that for a moment: you can read War and Peace in about 48 hours but in that same timeframe another 599,999 more have been created…

We’re now in a situation where there is too much information for people, and the tools we’ve developed, to process. Not so long ago it used to be prohibitively expensive to keep this much information. Storage is now available so cheaply, companies are able to save and use the data to gain a competitive edge.

Big Data is the term used to describe the new tools and processes that allow business to understand this information. It can help you solve specific issues, like security and fraud, or find new business opportunities by guiding your decision making.

Online retailers, for example, can track every product a customer looked at before making a purchase. This data allows far greater personalisation than a simple “users also bought…” system.

You and I could both buy the House of Cards boxset but we might have arrived at it for completely different reasons. You might like the political aspects whereas I’ll watch anything directed by David Fincher. Our preferences will be clear from the journey we took to arrive at the boxset. If we can understand what our customers are subconsciously telling us, we can increase total spend and total completed transactions.

The education industry, which traditionally works on a yearly feedback cycle, is beginning to see the benefits of realtime feedback. Online learning platforms can help educators understand their students by tracking their behaviour. An abundance of information can be used to spot trends and correlations which would otherwise go unnoticed.

We’ve never known how long a student takes to answer a question or whether they change their answer once they’ve read a subsequent question. This type of information is absolutely invaluable. It gives a insight into the way a student is thinking. It allows students to be treated as individuals.

Businesses have been aware of the benefits of analytics for some time, but Big Data has some key differences, encapsulated by the 3 vees:

“Volume”. We’re dealing with more data than ever before, but we can now analyse it all instead of just taking samples.

“Variety”. We used to deal with data that was very structured and fit nicely into a spreadsheet or database. Data now comes in a range of formats, from text, location data, images and tweets. It’s all valuable so we need to process it all the same.

“Velocity”. We can no longer produce reports once per quarter. Big Data allows us to use near-real time insights to be more nimble than our competitors.

It is not just having more data than the competition that will give you an advantage. Historically a lot of this information lives in silos in different parts of an organisation. The most valuable insights may only be found by consolidating the data and viewing it holistically. Companies need leadership teams that set clear goals and ask the right questions. True advantage will come by combining your domain expertise with data insights.

If you’re not listening to what your users are telling you, someone else will.