Why your big data strategy is a bust

Matt Asay

Gartner’s Nick Heudecker has always been a bit of a party-pooper.

The man who has done more to ridicule data lakes (a “fallacy”) and chastise the vendor-infested Open Data Platform (“for vendors, by vendors”) now has his sights set on big data, generally.

In retiring Gartner’s Hype Cycle for big data, Heudecker blithely reasons that “big data is no longer a topic unto itself” and needn’t be treated as such. This is similar to what happened with open source: For years it was a big topic, then it became standard operating procedure for any business serious about software.

But if big data has reached a point where it has simply become essential to everything else, why are so many companies still struggling to put their data to good use?

The big data furniture

Big data used to drive big page views and consulting fees, but now, argues Heudecker, “the various topics formerly encompassing big data evolved into other areas,” including the following:

• Advanced analytics and data science
• Business intelligence and analytics
• Enterprise information management
• In-memory computing technology
• Information infrastructure

If these sound like substitutes for the more malleable “big data,” in some ways they are. But I think Heudecker’s point goes broader.
Really, he’s arguing that big data isn’t something distinct that a company should, or can, do. Big data is not a strategy. It’s not a business objective.
Rather, big data is simply a way of describing data that is high volume or highly variable, or it moves at high velocity (or some combination of the three). Guess what? Most companies now have this, whether they’re a Google or a Kroger.

Such data, says Heudecker, is “no longer exotic,” but “common.”

As such, talking about big data as a meaningful category is far less interesting than talking about specific ways to put data to use within an organization. Those more interested in chasing buzzwords can turn their attention to the Internet of things, he mocks.

Everyone is doing it

Whether because enterprises persist in misunderstanding big data or for other reasons, meaningful impact from big data initiatives remains elusive for most organizations.

This seems surprising, given rampant interest in data, as a Teradata-funded Forbes report shows.

Forbes Insights-Teradata

This is partly a matter of skills shortage. As Heudecker highlights, “Thru 2018, 70 percent of Hadoop deployments will not meet cost savings and revenue generation objectives due to skills and integration challenges.”

In other words, companies would do more with their data if they had the people requisite to manage the data infrastructure. The real problem goes deeper, though it is tied to the skills shortage.

The CEO is the problem

The core issue is that it’s easier to say “big data” than it is to embrace big data. The former fits neatly into a PowerPoint presentation. The latter requires cultural change. As the Forbes report explains:

Part of the problem is that data-driven business models represent a break from the past that can call for a huge cultural upheaval. Almost overnight, employees must make data a priority, be willing to share data sets across departments and assume shared responsibility for data collection, quality and analysis.

A separate Booz Allen report uncovers the same reality: “The hard truth is that a key enabler to delivering on the biggest promise of data science is transforming organizational culture.”

Or as Michael Schrage, a research fellow at MIT Sloan School’s Center for Digital Business, insists:

[The] most enduring impact of predictive analytics ... comes less from quantitatively improving the quality of prediction than from dramatically changing how organizations think about problems and opportunities.

"Hard." "Huge." "Dramatic." These are the words used to describe the kind of change required to truly become data-driven. Most companies simply aren’t up to the task.

Have no fear, though: It’s the boss’s fault.

If you ask business leaders to name their strategic challenges, “making fact-based business decisions based on data” tops the list (48 percent of respondents to the Forbes survey). But when you ask them how willing they are to trust that data, a less rational picture emerges.

A Fortune survey of 720 senior business leaders that revealed that 62 percent tend to trust their gut rather than data, and 61 percent indicated real-world insight tops hard analytics when making decisions.

In other words, the problem with truly embracing big data starts at the top.

We have a long way to go, baby

This is why we’re seeing a boom in the adoption of such big data technologies as Hadoop, Spark, MongoDB, and Cassandra, even as returns on these investments sometimes stall. The problem isn’t the tools -- it’s the culture. And that culture starts with the C-suite.

Big data is big business for vendors of data infrastructure. As Mark Torr details, the Hadoop vendors are on a tear. This same observation holds true across the big data landscape. So companies are clearly spending on the big data dream.

But as erstwhile Wall Street analyst (and current Aerospike executive) Peter Goldmacher once declared, the big winners in big data won’t be the vendors. They’ll be the companies that leverage data “to create entirely new businesses or disrupt legacy businesses,” such as Uber or Facebook or any number of truly data-driven ventures.

To derive anywhere near that sort of value, enterprise CEOs will have to become as committed to acting on data as their developers and business analysts are about giving them access to that data through modern big data technologies. We have a long way to go.