Is AI the key to managing rampant data storage costs?

23 May 2019

We are currently living in a data economy, where most businesses' competitive advantage stems from its ability to analyse data, improve customer experience and decision-making.

Eran Brown, CTO for EMEA at Infinidat

As this data increases, due to the advent of emerging technologies and applications, it is simultaneously opening up additional possibilities. If data is at the core of our competitive edge, operating this data cost-efficiently and with high performance is the new business imperative.

Fortunately, Artificial Intelligence (AI) in the form of learning algorithms can help organisations improve data performance and mitigate the increasing cost of storing vast and ever-growing volumes of data.

Dealing with data the traditional way

The traditional approach to data storage is to place data that is currently being used in a high-performance production environment. This ecosystem is usually made up of a variety of different storage media, including Flash Array (AFA) and separate, isolated secondary storage arrays. However, by taking a conventional approach to storing data, the greater the performance required, the more expensive media is needed.

Paradigm shift in the storage environment

Eran Brown 5 Apr 2019

In a digital economy, this method poses a number of challenges. Firstly, it creates silos of data, as multiple solutions are deployed spanning a variety of media thereby increasing complexity and costs while reducing business performance. Moreover, each silo will have its own capacity, which cannot be accessed by any other silo. When more capacity is required it must be added in reactively, further driving up costs and due to the time taken to implement, reducing business responsiveness.

Secondly, it cannot easily cater for changes to production or mission-critical data. As a result, organisations will need to maintain large repositories of high-performance storage to ensure that all data that may be needed can be accessed quickly.

AI – delivering agility, performance and lower costs

The critical element that is missing from traditional data storage infrastructure is agility - the ability to respond quickly to change in demand. Until now, the only way to guarantee high-performance was to spend large sums of money on expensive storage tiers.

This is simply not a financially viable option given the cost involved, nor is it the most sophisticated approach. However, AI could be the missing link in this scenario and provide the required agility and assist IT managers to reduce their storage spend. So, how exactly can AI assist?

AI-driven learning algorithms can revolutionise the way businesses manage data storage. These algorithms respond in real time to dynamically changing workloads, supplying non-stop performance optimisation for your data.

Consequently, many Inputs/Outputs (I/Os) are then delivered via RAM, which is up to 100 times faster than flash. The result is that the business can maintain its data in cost-effective hardware, while obtaining speed and performance superior to that of AFAs, at a fraction of the cost.

But...how?

No organisation uses 100% of its data 100% of the time – data is only processed in subsets, and at any given second the amount of data requiring high performance is quite small. Yet, these subsets are constantly changing. Learning algorithms respond in real time to track what is being used and dynamically move the active data sets into RAM predictively and proactively, using AI to determine what data will be required based on past history.

The result is that virtually all data reads and writes can be processed using RAM, while the cost of storage is alleviated thanks to the ability to use low-cost disks without compromising performance.