in Up Front
print the content item

"It's complicated." That phrase properly describes the relationship between big data and utility companies today.

Big data is hardly new, as the proliferation of the smart grid, which escalated more than five years ago, brought in a flood of data - from a variety of sources that include smart meters, outage management systems, supervisory control and data acquisition history, and customer data and feedback - into legacy IT infrastructures that simply were not prepared to handle the influx.

Today, while utilities stand to gain significant benefits from big data, they are still facing formidable challenges - not only in how to collect the data, but also in how to manage and actually use it organization-wide. A scalable and efficient IT infrastructure can help utilities solve part of the equation, but utilities still need analytics professionals to make sense of this data deluge. And these are not problems unique to utilities. Many industries - from retail to communications to financial services - face the same issues.

So, what's the answer? How do we transform big data into big value?

From "information explosion" to big data analytics

In the past, data collection and analysis were traditionally reserved for those working in science. Today, however, smarter technologies are bringing in a flood of data across almost every industry. According to one estimate, by 2009, utilities already possessed 194 petabytes of data. With this volume of data at their disposal, companies realized they could derive unique insights from various corners of their businesses and gain valuable knowledge by combining that information to create a larger, holistic view of what was happening organization-wide at any given time. In other words, big data became mainstream when businesses learned that the whole is greater than the sum of its parts.

Now, virtually every industry is looking to leverage big data in some way. Utilities can use it to anticipate energy usage patterns and respond more quickly to outages during storms; retail stores can use it to anticipate consumer demand for products and calculate projected sales; and banks can use it to track consumer activity and identify fraud incidents. If you have ever received a call from your credit card company about suspicious charges or gotten an emergency weather alert on your cell phone, then you have experienced big data in action. Wouldn't it be great if your utility sent you a text if they suspected you had a water leak or an appliance that needed repair based on usage spikes analysis?

While companies are starting to realize the benefits of leveraging big data, they are also facing new challenges that require different ways of thinking.

So, how can utility companies take the next step with big data and analytics to realize new efficiencies organization-wide?

Minding the gap with a collaborative approach

As data management becomes more complex, utilities need IT and operations technology systems that enable easy integration and data sharing. Additionally, they need access to business intelligence tools that are tailored to their specific needs. These tools for statistical and advanced analysis must be able to work with distributed data to perform analysis regardless of where the data resides, scale as data volume grows and automate decisions based on analytical models.

Furthermore, to help ensure reliability, performance and security, utilities need a strong foundation of database, hardware and storage systems. All of these solutions should be flexible to enable utilities to implement what they need when they need it to meet their unique challenges head-on.

Technology presents utilities with excellent opportunities, including the ability to:

- Control costs and optimize limited budgets through better operational analytics and related investment planning tools.

- Improve service reliability through greater use of grid analytics, such as distribution asset risk assessment.

- Begin to test cloud services to address big data needs from converging technologies. The cloud offers a prime collaborative testing site for utilities to share analytical information and analytics best practices with assigned utility expert data scientists.

- Develop pricing programs for energy to either drive conservation or change customer usage patterns to maximize utilization of existing utility infrastructure.

Technology, however, is only half the battle - utilities also need the right people to do the job. Yet, a 2013 Oracle study, "Utilities and Big Data: Accelerating the Drive to Value," which surveyed more than 150 North American utility executives, found that fewer than one in three utilities say they have sufficient expertise regarding smart grid data analytics or data scientists. In other words, more than two-thirds of utilities see a skills gap within their company.

These "data scientists," as they are commonly called, must have a unique skill set to examine patterns in unstructured data. As such, utilities' investment in people is very important. Some ways utilities can address the skills gap today include the following:

Recruitment: Improving or increasing recruitment efforts is an obvious solution to talent issues, but that doesn't mean it's the easiest. There is significant competition across industries for data management and analytics specialists - due to increased demand for these skills - which has led many companies to recruit talent directly from the top big data analytics graduate programs. These programs offer master's degrees in marketing analytics, business analytics, information systems management and engineering, among others.

Training: Many utilities can (and are) training current employees to analyze data. Oracle's recent survey found that 90% of utilities who admitted to having a skills gap said they were training current employees to address the problem. Providing employees with training and growth opportunities is a smart investment for any organization, but it is especially useful when the organization needs specific skills, which is exactly what utilities are facing in terms of data analytics.

Analytics Services: Utilities can leverage collaborative analytics services to bridge the skills gap to drive immediate value of analytics without the upfront investment of systems and personnel.

Utilities can task these new analysts to mine data sources to identify areas to improve performance in crew efficiency, asset and grid reliability, asset maintenance and investment performance, as well as customer satisfaction.


Moving forward, it is important for utilities to recognize that there is no "silver bullet" for solving the skills gap and maximizing the potential of analytics. In the short and long term, utilities can change their big data relationship status from "It's complicated" to "In a relationship" by approaching it collaboratively - through people, process and technology. Only then will utilities realize big data's full potential - transforming information into actionable intelligence to improve operational performance across every aspect of their businesses.

Brad Williams is vice president of industry strategy at Oracle Utilities.

Hybrid Energy Innovations

Hybrid Energy Innovations 2015
Latest Top Stories

To Help Combat Costly Electricity Theft, Emerging Markets Turn To Smart Grid Infrastructure

According to a new report, the world loses a whopping $89.3 billion to electricity theft annually, and emerging country markets are feeling a majority of the pain.

Report: Utilities Face Big Revenue Cuts From Distributed Energy Resources

According to a new report from Accenture, the growth of solar and other resources, such as energy storage, could mean significant losses for utilities.

NextEra To Acquire Hawaiian Electric In $4.3 Billion Merger Deal

If approved, the agreement between the two energy companies would produce one of North America's largest renewable energy players.

EPA Racks Up Over 1.6 Million Comments On Carbon Pollution Plan

Months after the U.S. Environmental Protection Agency released its Clean Power Plan, which will create new regulations for existing power plants, the agency says it has received loads of feedback to consider.

Smart Grid Interest Continues To Spread Among U.S. Rural Utilities

A new survey gauges rural smart grid efforts across the country and offers key findings, one of which is that nearly all utilities polled are doing something to modernize their systems.

Hybrid Energy Innovations 2015