Simplifying big data with great visualization

Data visualization has an amazing ability to make the complex simple, and the latest tools can do much more than give everyone the same view of data. It’s only through visualization that we can take something as abstract as symbols and turn it into a physical image that has dimensions that our eyes can quickly see and our brains understand. We can grasp data’s meaning more quickly. Visualization tells us when trends are heading in the wrong direction and we need to intervene.

But even more, when we visualize the scope and scale of today’s enormous data sets, we pick up things with the naked eye that would otherwise be hidden. We can see data’s previously untold story. We can see where Pennsylvania’s Vote ID Law is disproportionately affecting minorities and students. Visualization of Twitter data can show us a revolution as it unfolds in Cairo. What was being reported in the media as mass crowds forming in Tahrir Square comes to life when the network data becomes visualized as it grows. The two images below show early Twitter traffic around the protests and then traffic as the word gets out that Mubarak is stepping down.

These are powerful images that tell a completely different story than an observer on the ground could make of the raucous crowds. The fascinating video is available on YouTube and is worth watching.

Increasing demands for visualization

With the amount of data already here and far more coming as we connect a predicted 50 billion sensors to the Internet (AKA, the Internet of Things), we have no choice but to rely on tools to sort all of that information and show the important parts intelligently. We need to continually find not just better data and faster computers, but also to find inventive ways to display data so that the nuances and hidden insights aren’t lost in tools that can’t manage nuance.

New kinds of visualized data

Beyond just data about supply chain or production statistics, we’re more often visualizing data that has many dimensions, like location data overlaid with precipitation or air pollution. Adding to the mix, time series visualizations show progression of different dimensions over time, unlocking enormous insights into how things happened, like erosion, flooding or retreating of glacial ice, not just what happened at a moment in time (that’s BI…). It’s Geovisualization and it brings an entirely new perspective to understanding Big Data.

One of our favorites has to be the wind map. This image is the real-time visualization of wind direction and strength across the US as this blog was being written. Click on the map to see for yourself what’s happening now, based on the National Digital Forecast Database.

Visualization simply works because it works simply

The power of being able to not just know, but to see data for what it really tells us is the point of visualization. The methods of display can’t be complicated or the point of visualizing data is lost. Visualization serves us because it puts the tools of understanding business directly in the hands of those needing to make decisions. And ultimately, making decisions is what Big Data is all about.

Oh, have we mentioned that we’ll be managing the Big Data Workshop at Interop in New York on October 1st? We hope you can join us there.