Solving the unstructured data puzzle

Lenore Kerrigan

The growing maturity around the Internet of Things, the massively increasing number of connected devices, the explosion of video, social media, user generated content have all resulted in exponential growth in the amount of data at our fingertips. In July 2015, YouTube reported that it was uploading 400 hours of video content every minute. So for every hour of YouTube videos you watch, you’re already 23,999 hours behind – that’s nearly three years!

The data challenge

From a business perspective, there are two types of data that need to be analysed, understood and trawled through; structured and unstructured. Analysing the structured data is a well-established business requirement – vast amounts of data stored in well-organised databases can be interpreted and presented with relative ease.

The challenge lies with analysing all the unstructured data residing in an organisation – from emails to voicemails, social media, video, contracts, letters. This unstructured information doesn’t clearly display any underlying patterns or trends. We also have to consider context. A contract, for example, is the final output. A negotiated, distilled and agreed-upon entity. But it is really just a snap-shot, a point in time. What about all the information that surrounds the contract, the email discussions the multiple revisions. What was actually meant by the parties negotiating the deal?

Historically, the only way to gain insight into a big stack of reports – or see patterns in customer complaint letters or supplier payment issues – was for a person to read through them manually and hope their knowledge, diligence or intuition picked up on common themes. However, in a digital world, relying on people alone isn’t going to cut it.

It could be argued that there is so much data, so much potential information that organisations are starting to suffer information-overload. Where do you start when you know you can never catch-up? Organisations need to change the rules and move from a traditional 'human-speed' approach to embracing 'machine-speed' only then will they become masters of this information.

Advances in analytics, semantic understanding and machine-learning has given organisations more power to interrogate this unstructured content in ways previously unavailable, transforming “I think” into “I know”. And increasingly, predictive analytics will start to play a part, transforming “I know what happened” to “I know what will happen.”

Marketing opportunity

As a marketer, this insight is incredibly valuable to me. Digital has transformed the way I can interact with my current and prospective customers, opening up a new world of possibilities for strategic marketing. Deep customer insight can be achieved not simply by assessing the browsing or shopping history, but by layering on social comments, or support call data, to develop a much more rounded, 360-degree view of the customer. It is now possible to combine real-time knowledge of customer information (identity, history, preferences) with customer context (presence, local conditions, bio-feedback) with predictive analysis to deliver the highest value experience to a customer.

There is so much data out there for companies to tap into. But this data is worthless unless it is transformed into information. By embracing unstructured data analytics marketers can improve campaign messaging, targeting ROI and customer satisfaction using data that already exists in their company’s internal and external systems. What’s more, they can do it far faster than their competitors who have not yet truly embraced digital.

Established organisations that make the most of the information will gain the necessary insights to stay ahead of the game in the digital-first world. Businesses need to embrace it now because if they don’t, they might find they are left behind.