The increasing cost of collecting and analysing today’s business data is placing corporations in New Zealand at risk for litigation through compliance breaches, and is negatively impacting their competitive potential in the wider market.

That’s according to a new IDC survey and whitepaper titled, “Driving Smart Data Management”, released today across the industry.

Commissioned by CommVault, Bryan Stibbard, Area Vice President ANZ claims organisations of all sizes and across all industries are missing huge amounts of value and businesses opportunities, as they continue to overlook the significance of analysing unstructured datasets.

“Unstructured data comprises the constant stream of information generated from everyday interactions via email and social media on mobile devices, to monitoring equipment, scientific research, and medical and government records,” Stibbard adds.

“The most critical element is being able to identify what that data is and where it resides within an organisation. That enables knowing what data should be kept and what should be discarded.”

A missed opportunity to gain business value from increasing volumes of captured data…

Stibbard believes there is a large discrepancy between the amount of data growth across New Zealand organisations and the level of analysis that is being applied, especially when considering analysis of semi-structured and unstructured data generated via text from communications systems.

The survey findings show that:

· 50 percent of all New Zealand organisations are expecting data growth as much as 20-50 percent in 2014, compared to an average of 39 percent across all of Asia Pacific that are expecting the same 20-50 percent data growth.

· Across all New Zealand organisations, 86 percent capture transactional data and only 73 percent of it will be analysed.

· Large discrepancies are seen between the capture and analysis of semi-structured and unstructured data generated via audio (43 percent vs. 25 percent) and video (53 percent vs. 35 percent).

"The real opportunity exists in combining internal sources of data, like customer data, with external sources like social networking and location,” says Daniel-Zoe Jimenez, IDC Senior Program Manager of Big Data and Analytics, Data Management, and Enterprise Applications (white paper author).

“This can help organisations gain a holistic view of their customer transactions and circumstances, enabling them to understand preferences, habits, and future requirements — all in context.

“ANZ organisations are among the most mature in the region, and they are now becoming more focused on utilising their data to the fullest extent possible, but the lack of skill sets, optimal tools, and gaps in processes still present challenges."

Delving deeper, the survey also found that Kiwi organisations are spreading critical business data across a combination of multiple locations, legacy physical systems and cloud-based services, adding to the complexity of ingesting, protecting and accessing information.

The whitepaper suggests that in order to derive value from data across these increasingly disparate tiers and physical locations, it is critical that data be transitioned into a single, consistent, vendor-agnostic and universally accessible store.

Further findings show that across New Zealand:

· 86 percent of organisations believe that a solution that can enable them to protect, manage and access all data using a single platform can allow them to 1) improve storage and network optimisation and cost savings, 2) increase protection from data loss and 3) enable more efficient and reliable disaster recovery.

· 46 percent of all companies have stated a preference for an end-to-end solution that allows protecting, managing and accessing all data from one single platform.

"Data management continues to be a top challenge for New Zealand organisations, especially in the context of Big Data and cloud,” Jimenez adds.

“All this demands new approaches to managing, finding, and analysing data across different sources. Having a holistic view into all data assets across applications, devices, operating systems and locations is critical.”

Stibbard adds that of the Kiwi firms that have approached CommVault to solve such challenges in the past six months include Computer Concepts Limited, Revera and Wanganui District Council.

“An effective data management solution needs to provide effective backup, replication, snapshots and archiving of both physical and virtual information, regardless of where that data resides,” he adds.

“This kind of single unified solution would not only alleviate customer concerns around how to effectively manage and protect data that is created outside the firewall or lives in the cloud, but it would also provide key insights that facilitate business decisions.”