Investment in big data Is up but fewer organizations plan to invest: Gartner

Big data investments continue to rise but are showing signs of contracting, according to a recent survey by Gartner, Inc. The survey revealed that 48 percent of companies have invested in big data in 2016, up 3 percent from 2015. However, those who plan to invest in big data within the next two years fell from 31 to 25 percent in 2016.

The online survey was conducted in June 2016 among Gartner Research Circle members. In total, 199 members participated and shared their investment plans.

"Investment in big data is up, but the survey is showing signs of slowing growth with fewer companies having a future intent to invest," said Nick Heudecker, research director at Gartner. "The big issue is not so much big data itself, but rather how it is used. While organizations have understood that big data is not just about a specific technology, they need to avoid thinking about big data as a separate effort."

Big data is a collection of different data management technologies and practices that support multiple analytics use cases. Organizations are moving from vague notions of data and analytics to specific business problems that data can address. "Its success depends on a holistic strategy around business outcomes, skilled personnel, data and infrastructure," added Mr. Heudecker.

Getting Big Data Projects to Production Is a Challenge

While nearly three quarters of respondents said that their organisation has invested or is planning to invest in big data, many remain stuck at the pilot stage. Only 15 percent of businesses reported deploying their big data project to production, effectively unchanged from last year (14 percent).

"One explanation for this is that big data projects appear to be receiving less spending priority than competing IT initiatives," said Mr. Heudecker. Only 11 percent of respondents from organisations that have already invested in big data reported that their big data investments were as important, or more important, than other IT initiatives, while 46 percent stated that they were less important.

"This could be due to the fact that many big data projects don't have a tangible return on investment (ROI) that can be determined upfront," added Mr. Heudecker. "Another reason could be that the big data initiative is a part of a larger funded initiative. This will become more common as the term "big data" fades away, and dealing with larger data sets and multiple data types continues to be the norm."

A further factor to consider is the lack of effective business leadership or involvement in data initiatives. Too often, pilots and experiments are built with ad-hoc technologies and infrastructure that are not created with production-level reliability in mind.

"When it comes to big data, many organisations are still finding themselves at the crafting stage," said Jim Hare, research director at Gartner. "Industrialization — and the performance and stability guarantees that come with it — have yet to penetrate big data thinking."