Looking beyond real-time

This week it was announced that a collaboration between Microsoft and the Technion-Israel Institute of Technology created software that predicts future events using a combination of archived news material from the New York Times, real-time data and information from a variety of web sources like Wikipedia.

In their research paper they focused on predicting disease outbreaks, riots and other public events and claimed their accuracy was between 70%-90%. They claim to be able to see links between droughts and storms in parts of Africa and cholera outbreaks from these disparate sources of new and old data. While previous research has been done in this area (see excellent Wired article on the subject from 2011) none have focused on using the data to try to look ahead and forecast future events before they happen, they said.

“I truly view this as a foreshadowing of what’s to come,” said Eric Horvitz, Codirector at Microsoft Research. “Eventually this kind of work will start to have an influence on how things go for people.”

But there’s more at stake here than just predicting the next public disturbance or disease outbreak, this has rather interesting commercial implications on a number of levels.

The rise of the digital fortune teller

As I wrote last year, you’ll soon be able to ask for an entire dump of what every social site holds on you into a cloud-based repository. Obviously what you’ve put into these social sites, loyalty cards, location-based apps, etc., is what you expect to receive back but the sheer amount of data processed, correlated and extrapolated will vastly outweigh what you put in in the first place. The sites hosting this information have all made use of it first.

A quick Google search will reveal a small advert for digital fortune telling, an offering to reprocess all that data so you, yes YOU, can make smart use of it. You will want the power that all those sites and companies held over you for so long. You want to be able to know your future before you make it. And they can offer you the chance to predict your future with it the same way the Microsoft and Technion team have just demonstrated. Will you cross their palm with silver to find out?

Real-time was yesterday’s dream

Predictive analytics and statistical modeling is nothing new in the enterprise, they are used every day in businesses like insurance and financial services. But they are used to drive commercial decisions based on a product level, for example targeted consumer marketing using data mostly held within the company. Once you begin to mine and correlate information from external sources at the same level shown in the research paper you have the ability to factor future events that may completely alter the behavior of the consumer, allowing the business to prepare to scale on a future demand that snaps into play with little to no warning. Reactive decision making will be a thing of the past. Real-time and Big Data seem like yesterday’s dream already.

The real power lies in making business decisions that are outside of the consumer insight sphere. Would you trust making one based on a future event prediction and bet your company budget on it ?

Can you imagine if Oreo had already predicted the blackout and instead of having a command center within their digital agency partner ready to react to any event as it unfolded they had already prepared for it because they predicted it?

I predict many will baulk at this idea. But the reality for us all may soon be very different.