Standing in the fire hose of data: drowning or refreshing?

Republished article on last year’s TUCON 2012 event. TUCON 2013 takes place October 14-17 at the Aria Resort in Las Vegas.

Big Data is a great topic. Everyone recognizes the rise of the volumes of data being generated by individuals, systems and devices. There are some fabulous headline figures about data. It is the journalists’, analysts’ and bloggers’ dream.

IT vendors are jumping onto the bandwagon and any analysis and reporting capability they have is being rapidly re-architected to be able to cope with the dramatic increase in data volumes. Some of this is still marketing-ware.

Big Data – So What?

What is far more interesting is the discussion that was a major part of the keynote at TUCON 2012: “What opportunities does Big Data open up?”

The strapline of the conference was “Everything is different” and the walls of the Aria Hotel in Las Vegas were littered with some compelling statistics about the exponential rise in data volumes. The idea is “while you were sleeping,” but of course, we’re in Las Vegas, so sleep is a secondary item!!

However, some of the numbers are staggering. While you were sleeping…

• 294 billion emails were sent
• 35 million apps were downloaded
• more iPhones were bought than babies born
• 250 million photos uploaded to Facebook
• 2 million blog posts were written
• information consumed on the internet would fill 168 millions DVDs

Big Data Goes to Work

The numbers are almost incomprehensible and they require different approaches to drive any value out of the stream of data. There is simply too much to capture, store in a database and later run reports. That is 20th-century thinking.

The exponential rise in data requires different approaches and different thinking. We need to analyse the “data in motion,” on the fly, in real time. We need to glean some insights as they pass, and much more if they are only valuable “in the moment.”

Real Companies, Real Examples

I had the pleasure of interviewing each of the keynote speakers at TUCON for some video case studies. While they covered different industries and a wide range of sizes of companies, there was been a common theme. They are all harnessing the stream of data, making sense of it, and using insights from the data to be able to transform their company’s products and services. They are able to offer what Vivek Ranadivé, TIBCO’s founder and CEO, calls “extreme value” – something that differentiates the star performers from the also-rans.

Here are just two examples:

Rick Welts, Golden State Warriors

Rick Welts is President and COO of the Golden State Warriors, the Bay Area basketball team. His entire career has been in the NBA and he is focused on using the different streams of customer data to dramatically improve the experience of going to a game. Interestingly, the enjoyment of the experience is not tied to the result of the game, which is lucky because in the past, the Warriors lost more than they won. But that is changing.

They look at the entire end-to-end process, from when you leave your house, park on the grounds, watch the game, get refreshments, leave the grounds and get home. The question they are asking is “How can we improve every facet of that end-to-end experience?”

Technology has a vital role to play and TIBCO is helping them with analysis of the data, and also the development of a mobile app that will enhance the fan’s connection with the team and enjoyment at the game.

Rick’s passion and enthusiasm for basketball, and the potential opportunities to leverage data and technology were infectious. I hope that comes over in the video footage.

Tom Siebel, C3

Tom Siebel is best known in technology circles as the founder and CEO of Siebel Systems, which was sold to Oracle in 2006 for $5.9 billion. Siebel made enough to find work was no longer necessary. But, like most entrepreneurs, he is driven and in 2009, he launched C3, an energy management company. He saw the opportunity to capture all the data being generated by sensors in companies and homes. C3 offers a family of software solutions that help companies to understand, optimize, and report on their energy use and greenhouse gas emissions, in order to reduce cost, risk, and environmental impact.

C3 is a visionary Big Data company, considering it was launched years before the term Big Data was coined.

C3’s SaaS-based software is driving incredible savings for companies, including the recent deal announced with Pacific Gas & Electric.

We see significant value for our commercial and industrial customers from using C3′s energy and emissions management solution. With C3, customers can not only better understand the details of their use, but PG&E can also identify the most appropriate high value mitigation projects for rapid energy savings.

- Saul Zambrano

As Tom said, no longer do you need huge financial backing to be able to launch a software company, unlike the days when he founded Siebel Systems. With SaaS technology platforms available that are paid based on consumption, the start-up costs can be low. The challenge is the integration of multiple streams of data from systems both inside his clients, but also from third-party systems. That is where his long relationship with TIBCO comes in. Siebel Systems and TIBCO had a long relationship, and so Tom again turned to TIBCO for help on the really tough integration issues he faced with his clients.

What was refreshing was that Tom’s vision was to build a significant company over the next decade. He recognizes that value takes time to build. He and the other panelists at TUCON (Vivek Ranadivé, Chairman and CEO, TIBCO; Vikram Pandit, CEO of Citi; Scott McNealy, Co-Founder of Sun Microsystems and Chairman of Wayin and K.R. Sridhar, Principal Co-Founder and CEO of Bloom Energy) were wonderfully dismissive of over-hyped social media companies with sky high, but rapidly dropping valuations.

Big Data Opens Up New Opportunities

So Big Data is allowing existing companies to deliver “extreme value.” It is a massive competitive differentiator, but it does require discontinuous, visionary thinking. The technology is available from TIBCO to make sense of the data, but the new opportunities will require reshaping their companies. That means process redesign and re-engineering. TIBCO technology is also available to do that.

For start-ups, there are huge opportunities to exploit Big Data, enter the market and sidestep the incumbent players. And with cloud-based applications paid on a consumption basis, the barriers to entry have never been lower.

We live in exciting times. Let’s dive in.