The reason AI can predict consumer behaviour

Vian Chinner, CEO at Xineoh

If you think the above sounds like a bad joke or a riddle that cannot be solved by any human being, not even the smartest actuarial or data scientists around, you are not alone. Where would you even start to make sense of these two apparently disparate shoppers going about their business in different parts of the country, in different seasons, at different ages and levels of fitness? There are just too many dimensions to consider.

Two dimensions are in most cases the limitation of what humans can understand. Perhaps we can comprehend the complexities up to three dimensions, but by the time we get to four it becomes almost impossible. Artificial Intelligence (AI), however, can go to a thousand dimensions easily.

Give an AI platform enough data (eg. information about shoppers, their ages, gender, geographical location, their choice of products and the time of purchase), and it will – with remarkable accuracy – be able to predict the missing item from our hypothetical shopping basket.

Possibilities

To make the example a bit more digestible to the man on the street: You would most likely know the answer if the riddle was simpler. King is to queen as man is to...? Intuitively most people would understand that the underlying two dimensions to this problem are power and masculinity. To get to the answer you subtract power from the king to get to man, and then (remaining on that power level) you move away from masculinity towards... woman. Voila, two-dimensional problem solved.

AI has broken open the possibilities of what we can predict in terms of consumer behaviour simply because it has the capacity to very quickly, and on a continuous learning curve, make sense of multiple dimensions all at once and group consumers in a way that would not instinctively make sense to us.

But how is it possible? What ultimately binds human behaviour in such a way that makes it possible for machine learning to decipher the building blocks of why we do things the way we do them, and then predict future behaviour?

To get to this answer you must try to marry mathematics and philosophy. Or at least ask them to sit at the same table and have a discussion.

"Collective subconscious"

The philosopher Carl Jung first coined the phrase "collective subconscious". He believed that this consists of structures of the subconscious mind that is shared among beings of the same species. In our case – humans.

He argued that the collective subconscious is made up of instincts and archetypes (primordial, structural elements or the essence of every human's psyche) that is common to all of man and responsible for a number of deep-seated beliefs and, ultimately, behaviours.

He believed that intuitively all people also attach the same meaning to these archetypes, even across language and culture. It doesn't matter where you live, what language you speak or what your religion you claim as your own – if you hear the concepts hero, mother or jester (amongst other identifiable archetypes), we all instinctively attach the same meaning to it.

Jung's ideas were based substantially on what Plato argued centuries before, when he developed his argument of Forms. Crudely (and this could be a complete thesis on its own) Plato argued that the Form of something is the element without which that particular object would not be the thing it is – again, the essence.

Reconstruct the building blocks

For the purposes of how it links to AI, we can look at it as the most basic building blocks that make up an object, a behaviour, even a choice. In our daily world these building blocks are piled on top of each other and layered under multiple dimensions of complexity.

AI has the unique capability to start at any part of the cycle of complexity and break it down into its most basic elements (the building blocks) and also reconstruct those basic elements into other complex concepts. It can break down the specific consumer event mentioned in our riddle (a sporty man, aged 26, in Durban, in July, buying a bowtie and board short) into its most basic parts (gender + lifestyle activity + season + geographical location + formal men's wear + leisure wear + price) and through machine learning build it back up and predict, with surprising accuracy, future shopping behaviour where one or two of the dimensions could be missing.

Our brains don't understand how this could be possible. Simply because we cannot comprehend past three dimensions. It is just that we cannot have sight of complex interactions or objects that are made up of many different dimensions and understand the most simple basal elements it is made up of. Then, on top of that, we will surely not be able to sort them in order to start to uncover the type of consumers that exist.

Predicting human behaviour

But AI can. It clusters, classifies and predicts where the human brain would not be able to.

What philosophers like Plato and Jung, however, realised long before AI was an acronym, was that spanning all the differences present in humans (whether those are apparent in the individual's consciousness or hidden in the subconscious), is that layer of the collective subconscious that binds us together.

If that layer did not exist, there would be no similarities in behaviour for AI to discover at the very core of the multiple dimensions. Everything would be completely random and diffused. AI can only discover these building blocks of human behaviour and utilise them for business insight because a collective subconscious links us all. It is due to those symbols, models or archetypes that form part of an innate understanding that we all share.

Consumer prediction companies can utilise AI and machine learning algorithms that tap into the building blocks of human behaviour to predict future consumer actions. They extract useful insights from the data provided to come up with solutions for businesses. First the data is broken into the building blocks, then sorted into new archetypal consumers or clusters of products that behave similarly and then the insights are applied to make operations more efficient.

Using the insights gained, for example, a store can stock only 50% of all available items at any given time – ensuring that it has them in the right store, at exactly the right time – and still achieve close to 100% of possible sales. So half of the stock, same level of revenue.

The only way it is possible to know what a specific consumer will choose at a specific time or season or life stage, is because of the existence of those shared essential elements and building blocks that guide human behaviour and that can be deciphered using AI.