Why AI, ML are vital to tackling the data explosion

26 Mar 2019

In 2019, it's estimated we'll generate more data than we did in the previous 5,000 years. Data is fast becoming the most valuable asset of any modern organisation, and while most have access to their internal data, they continue to experience challenges in deriving maximum value through being able to effectively monetise the information that they hold.

Zakes Socikwa

The foundation of any analytics or Business Intelligence (BI) reporting capability is an efficient data collection system that ensures events/transactions are properly recorded, captured, processed and stored. Some of this information on its own might not provide any valuable insights, but if it is analysed together with other sources might yield interesting patterns.

Big data opens up possibilities of enhancing internal sources with unstructured data and information from Internet of Things (IoT) devices. Furthermore, as we move to a digital age, more businesses are implementing customer experience solutions and there is a growing need for them to improve their service and personalise customer engagements.

The digital behaviour of customers, such as social media postings and the networks or platforms they engage with, further provides valuable information for data collection. Information gathering methods are being expanded to accommodate all types and formats of data, including images, videos, and more.

In the past, BI and data mining were left to highly technical and analytical individuals, but the introduction of data visualisation tools is democratising the analytics world. However, business users and report consumers often do not have a clear understanding of what they need or what is possible.

AI now embedded into day to day applications

To this end, artificial intelligence (AI) is finishing what business intelligence started. By gathering, contextualising, understanding, and acting on huge quantities of data, AI has given rise to a new breed of applications - one that’s continuously improving and adapting to the conditions around it. The more data that is available for the analysis, the better is the quality of the outcomes or predictions.

In addition, AI changes the productivity equation for many jobs by automating activities and adapting current jobs to solve more complex and time-consuming problems, from recruiters being able to source better candidates faster to financial analysts eliminating manual error-prone reporting.

This type of automation will not replace all jobs but will invent new ones. This enables businesses to reduce the time to complete tasks and the costs of maintenance, and will lead to the creation of higher-value jobs and new engagement models. Oracle predicts that by 2025, the productivity gains delivered by AI, emerging technologies, and augmented experiences could double compared to today’s operations.

According to the IDC, worldwide revenues for big data and business analytics (BDA) solutions was expected to total $166 billion in 2018, and forecast to reach $260 billion in 2022, with a compound annual growth rate of 11.9% over the 2017-2022 forecast period. It adds that two of the fastest growing BDA technology categories will be Cognitive/AI Software Platforms (36.5% CAGR) and Non-relational Analytic Data Stores (30.3% CAGR)¹.

Informed decisions, now and in the future

As new layers of technology are introduced and more complex data sources are added to the ecosystem, the need for a tightly integrated technology stack becomes a challenge. It is advisable to choose your technology components very carefully and always have the end state in mind.

More development on emerging technologies such as blockchain, AI, IoT, virtual reality and others will probably be available on cloud first before coming on premise. For those organisations that are adopting public cloud, there are opportunities to consume the benefits of public cloud and drive down costs of doing business.

While the introduction of public cloud is posing a challenge on data sovereignty and other regulations, technology providers such as Oracle have developed a ‘Cloud at Customer’ model that provides the full benefits of public cloud - but located on premise, within an organisation’s own data centre.

The best organisations will innovate and optimise faster than the rest. Best decisions must be made around choice of technology, business processes, integration and architectures that are fit for business. In the information marketplace, speed and informed decision making will be key differentiators amongst competitors.