Big Data is the greenest data of all

The following is a guest post. As Co-founder of Fabless Labs, Piyush is in his element while developing the next generation of truly open, open source analytics solutions.Piyush is likewise no stranger to the challenges of real-time and complex information management. For him, this is the perfect intersection of his passion for real-time decision-making and the rapid transformation taking place in the marketplace.

Green is the new black…or so you’d think from the incredible amount of focus paid on efficient energy production and consumption. With so much emphasis on building a green planet and increased government and utility spending on energy efficiency, one would expect a reduction in total energy spend in commercial and residential markets. It hasn’t happened.

Despite the recent push to implement energy efficiency programs, total energy consumption and average energy prices in both markets continues to rise. Consumers continue to face higher energy bills because the average energy price is rising faster than the reduction in customer demand. These indicators might make you think we’re going in the wrong direction, but it is actually too early to say all of this effort isn’t working.

Give it time

The reason for optimism? Smart meters and a variety of sensors across the energy supply chain are creating the ability to collect and analyze massive amounts of energy production, transmission and consumption data. The arrival of Hadoop and other Big Data tools makes it possible for analysis to keep up with rapidly increasing data volumes. All of that means nothing if it isn’t actionable. Let’s take a look at just a couple of ways that Big Data can be Green Data.

• Forecasting demand - We’re slowly moving toward homes and businesses having smart meters that report actual consumption back to utilities and allow decisions on how to supply energy more efficiently. Right now, those meters provide data in 15-minute increments but down the road, we can increase the frequency of reporting as the ability to take and crunch data increases. When we know more, we make better energy supply decisions.
• Conservation ‘signals’ - To make markets behave efficiently, there needs to be a way for energy use to change with availability. The most common way this is done is through price. Once energy providers can accurately forecast demand and in-the-moment use, pricing signals will cause consumers, individual or commercial, to lower usage. This proactive approach is mostly missing from today’s energy markets. When we know more, we make more efficient usage decisions.
• Measuring efficiency - The US Department of Energy is currently developing the SEED database, meant to be a way to allow buildings to benchmark against other facilities. While that may not seem so hard, there are big factors that need to be part of any efficiency algorithm, like weather, the number of people, what types of machines are being operated and when. Once buildings have an ‘energy value’, decisions can be made on how much real estate is worth, where to retrofit and how to design new facilities. When we know more, we can make better design decisions.

Change needs to happen

This is all great and in theory, will make our planet better for everyone. There are still a few things, however, that stand in the way. There needs to be better standards for how information is collected, stored and shared so that energy supply chains can be better analyzed and operated optimally. We also need to make sure energy providers aren’t reaping excessive benefit from more efficient consumption without passing that benefit to energy consumers, which would ‘mute’ the signals that drive positive change. Balancing the right amount of regulation with allowing market forces to operate is an age-old challenge.

Assuming we’ll overcome these challenges, where things go is wide open. There are gamification possibilities (who’s the most green in the neighborhood/city/region?) and countless other strategies on the horizon. There’s no doubt Big Data will be driving us toward a greener planet.

Google+

Christopher Taylor